Three faces of the second law. I. Master equation formulation
Massimiliano EspositoCenter for Nonlinear Phenomena and Complex Systems, Université Libre de Bruxelles, CP 231, Campus Plaine, B-1050 Brussels, Belgium
Christian Van den Broeck
Faculty of Sciences, Hasselt University, B-3590 Diepenbeek, Belgium
共Received 10 May 2010; published 30 July 2010兲
We propose a formulation of stochastic thermodynamics for systems subjected to nonequilibrium constraints 共i.e. broken detailed balance at steady state兲 and furthermore driven by external time-dependent forces. A splitting of the second law occurs in this description leading to three second-law-like relations. The general results are illustrated on specific solvable models. The present paper uses a master equation based approach.
DOI:10.1103/PhysRevE.82.011143 PACS number共s兲: 05.70.Ln, 05.40.⫺a
I. INTRODUCTION
The second law of thermodynamics specifies that the total entropy of an isolated macroscopic system cannot decrease in time. This statement applies to the stages of the evolution in which the entropy is well defined. For example, for a system in equilibrium at initial and final times, the final en-tropy will be larger than the initial one, even though the entropy may not be well defined during the intermediate evo-lution. However, it is often a very good approximation to assume that the system is in a state of local equilibrium, so that the entropy is well defined at any stage of the process. For example, linear irreversible thermodynamics is built on such an assumption, allowing the use of the Gibbs relation to define entropy locally in terms of the slow conserved quan-tities共for example, momentum, energy, and concentration of the constituents兲 关1–3兴. The second law can then be reformu-lated as the non-negativity of the irreversible entropy pro-duction共EP兲 S˙i共t兲ⱖ0 关4,5兴,
S˙共t兲 = S˙e共t兲 + S˙i共t兲, 共1兲 where S˙共t兲 is the entropy change of the considered subpart and S˙e共t兲 is the entropy flow to the environment. In some cases, the environment is idealized as being one or more reservoirs without internal dissipation, so that their entropy change is equal to minus the exchange term: S˙r共t兲=−S˙e共t兲. For a single heat reservoir, this entropy exchange is given by the energy inflow divided by its temperature. In the sequel, we will be mainly interested in this situation, with the sub-system of interest, henceforth called the sub-system, in contact with an environment consisting of one or more ideal reser-voirs. The irreversible EP in this system is then equal to the total EP S˙tot共t兲⬅S˙i共t兲ⱖ0.
In more recent developments关6–33兴, it has been realized that one can formulate thermodynamics for small systems incorporating the effect of the fluctuations. These develop-ments can be seen as the continuation of the pioneering work started by Onsager and co-workers关34–37兴, with as interme-diate steps the fluctuation-dissipation theorem 关38兴, the theory of Gaussian stochastic processes and linear response 关39,40兴, and Green-Kubo relations 关41,42兴. The essential
in-gredient is to guarantee the consistency of the statistical ir-reversible laws on the system dynamics with the reversibility of the equilibrium reservoirs statistics. In this new thermo-dynamics, also called stochastic thermodynamics关43,44兴, the system is described by a probability distribution pmevolving according to a Markovian master equation. The exchange of energy共heat兲 or particles with the environment and the other thermodynamic quantities associated to the system states m become stochastic variables. The rates associated to each res-ervoir satisfy the property of local detailed balance reminis-cent of the fact that they always remain at equilibrium. The system entropy is defined using the Shannon expression S = −⌺mpmln pm and entropy balance equations of the usual form can be derived via the identification of a non-negative EP consistent with macroscopic nonequilibrium thermody-namics. A minimum EP theorem can also be proved 关6,8,43,45–47兴.
efficiency of thermal machines at maximum power 关50,51兴. Very much in the spirit of the fluctuation theorem, Jarzynski 关13,14兴 and Crooks 关16,20,21兴 obtained the work theorem 共see also 关52–54兴兲. They found that the probability distribu-tion for the work w performed on a driven system, initially in canonical equilibrium at a specific inverse temperature −1, obeys the relation P共w兲/ P¯共−w兲=exp兵共w−⌬F兲其. The over-bar corresponds to the probability distribution for the time-reversed experiment. If one assumes that, at the end of the driving, the system relaxes back to equilibrium 共at inverse temperature −1兲 then 共w−⌬F兲, the so-called dissipated heat, is equal to the change in total entropy ⌬stot of system plus reservoir, so that the work theorem becomes a fluctua-tion theorem for the change in total entropy ⌬stot, P共⌬stot兲/ P¯共−⌬stot兲=exp共⌬stot兲 关23,25兴. Note that this result is valid for all times. The previous asymptotic fluctuation theorem can in fact be seen as a special case if one assumes that, aside from an initial transient, the steady state can be maintained for long enough times by appropriate driving, so that ⌬stot=⌬s+⌬sr⬇⌬sr. The focus on the asymptotic form and the accent on large deviation properties are in our opin-ion a somewhat misleading representatopin-ion of the fluctuatopin-ion theorem. This form arises from the neglect of the system’s entropy共the so-called boundary term兲, for which no readily acceptable interpretation was deemed to exist at the time of the first formulations of the fluctuation theorem. Further-more, the validity of the theorem is typically compromised for a system with unbounded energy 关55–59兴, which is of course more of the rule rather than the exception. We note however that current fluctuation theorems关27,28兴 do require the long-time limit since currents are related to the⌬srpart of the entropy.
A further advance consisted of the formulation of the equivalence of the second law at the stochastic level. This required the identification of a trajectory-dependent system entropy. Even though the idea is well known in information theory, where −ln pmis the surprise at observing outcome m when its probability is pm, it took some time before Seifert 关25兴 identified this quantity as the appropriate stochastic sys-tem entropy, s共t兲=−ln pm共t兲共t兲. Note that it depends on the actual state m共t兲 of the considered trajectory at the consid-ered time, as well as on the probability for this state, which itself is in general time dependent. By taking this term into account, the asymptotic fluctuation theorem could be re-placed with a fluctuation theorem for the total entropy, which is valid for all times, just as the work theorem of Jarzynski and Crooks. For driven systems in contact with a single res-ervoir and subjected to a nonconservative force keeping the steady states out of equilibrium, Oono and Paniconi 关60兴 discussed an alternative way of splitting the EP by introduc-ing the excess entropy and housekeepintroduc-ing heat. This led to the formulation of two other fluctuation theorems, namely, the one derived by Hatano and Sasa 关22兴 共for system entropy plus excess entropy兲 and the other one derived by Speck and Seifert关61兴 共for the housekeeping heat兲. While both assume single reservoirs, the former is further restricted to transitions between nonequilibrium steady states.
To close this introductory discussion, we turn to a recent development关33兴 共see also 关29–31,62,63兴兲, which provides a
clarifying and unifying approach of the various fluctuation and work theorems. The total stochastic EP⌬stotis the sum of two constitutive parts, namely, a so-called adiabatic ⌬sa and a nonadiabatic ⌬sna contribution. Each of these contri-butions corresponds to the two basic ways that a system can be brought out of equilibrium: by applying steady nonequi-librium constraints 共adiabatic contribution兲 or by driving 共nonadiabatic contribution兲. Note that the term “adiabatic” is used here, not in its meaning referring to the absence of heat exchange, but in the meaning of instantaneous relaxation to the steady state. The crucial point is the observation that each of these contributions obeys a separate fluctuation relation, namely关33兴, P共⌬stot兲 P ¯ 共− ⌬stot兲 = e⌬stot, 共2兲 P共⌬sna兲 P ¯+共− ⌬s na兲 = e⌬sna, P共⌬sa兲 P+共− ⌬sa兲 = e⌬sa. 共3兲 The superscript + refers to the adjoint dynamics共also called dual or reversal 关21,62兴兲. The aforementioned fluctuation theorems of Hatano and Sasa and of Speck and Seifert are special cases of the fluctuation theorem for ⌬sna and ⌬sa, respectively, when single reservoirs and transitions between steady states are considered. To stress the special status of these theorems, we notice that they arise because of the two available operations to gauge the amount of time-symmetry breaking, namely, time reversal of the driving 共overbar: −兲 and the time reversal of the nonequilibrium boundary condi-tions 共superscript: +兲. Applying each of them separately, or both, leads to the three different contributions for the EP.
The above fluctuation theorems imply that the total, adia-batic, and nonadiabatic entropy changes have to be non-negative, each taken separately,
⌬Stot=具⌬stot典 ⱖ 0, 共4兲 ⌬Sna=具⌬sna典 ⱖ 0, ⌬Sa=具⌬sa典 ⱖ 0. 共5兲 This suggests that the second law can in fact be split in two. There are thus three faces to the second law: the increase in the average total entropy, the increase in the average adia-batic entropy, and the increase in the average nonadiaadia-batic entropy. Our purpose here is to clarify and document further the physical properties and the meaning of this remarkable result. In this paper we will focus on the implications for a description in terms of a master equation. The next paper 关64兴 deals with the corresponding results for Langevin and Fokker-Planck dynamics.
II. MASTER EQUATION A. Entropy balance
p˙m共t兲 =
兺
m⬘Wm,m⬘pm⬘共t兲, 共6兲 where the rate matrix satisfies
兺
m Wm,m⬘= 0. 共7兲 The transitions between states m can be due to different mechanisms . Furthermore, these rates can be time depen-dent via a control variable. We thus haveWm,m⬘= Wm,m⬘共t兲 =
兺
Wm,m共兲 ⬘共t兲. 共8兲 When the control variabletchanges in time we say that the system is externally driven. For rates that are “frozen” at the values Wm,m共兲 ⬘共兲, there is a corresponding stationary distribu-tion, pm
st共兲, which we suppose to be unique 共i.e., the rate matrix is irreducible兲 and which will always eventually be reached by the system. It is given by the normalized right eigenvector of zero eigenvalue of the transition matrix,
兺
m⬘
Wm,m⬘共兲pmst⬘共兲 = 0. 共9兲 Let us now investigate the time dependence of the system’s Shannon entropy共Boltzmann constant kB= 1兲,
S共t兲 = −
兺
mpm共t兲ln pm共t兲. 共10兲 Using Eqs.共6兲 and 共7兲 and omitting for compactness of no-tation the dependences of pmon t and of W ont, we find
S˙共t兲 = −
兺
m p˙mln pm = −兺
m,m⬘, Wm,m共兲⬘pm⬘ln pm pm⬘ =1 2m,m兺
⬘,兵Wm,m⬘ 共兲 pm⬘− Wm⬘,m 共兲 pm其ln pm⬘ pm =1 2兺
m,m⬘, 兵Wm,m⬘ 共兲 pm⬘− Wm⬘,m 共兲 pm其ln Wm,m共兲 ⬘pm⬘ Wm共兲⬘,mpm +1 2兺
m,m⬘, 兵Wm,m共兲 ⬘pm⬘− Wm⬘,m 共兲 p m其ln Wm共兲⬘,m Wm,m共兲⬘. 共11兲 It is revealing to introduce the fluxes Jm,m共兲 ⬘and corresponding forces Xm,m共兲 ⬘, Jm,m共兲⬘共t兲 = Wm,m共兲 ⬘共t兲pm⬘共t兲 − Wm⬘,m 共兲 共 t兲pm共t兲, 共12兲 Xm,m共兲⬘共t兲 = lnWm,m⬘ 共兲 共 t兲pm⬘共t兲 Wm共兲⬘,m共t兲pm共t兲 . 共13兲We can rewrite the master equation 共6兲 as
p˙m共t兲 =
兺
m⬘, Jm,m共兲 ⬘共t兲 =兺
m⬘ Jm,m⬘共t兲, 共14兲 where we defined Jm,m⬘共t兲 =兺
Jm,m共兲 ⬘共t兲. 共15兲 The system’s EP can thus be rewritten under the familiar form of irreversible thermodynamics,S˙共t兲 = S˙e共t兲 + S˙i共t兲. 共16兲 The quantity S˙e共t兲 = 1 2
兺
m,m⬘, Jm,m共兲 ⬘共t兲ln Wm共兲⬘,m共t兲 Wm,m共兲 ⬘共t兲 =兺
m,m⬘, Wm,m共兲 ⬘共t兲pm⬘共t兲ln Wm共兲⬘,m共t兲 Wm,m共兲 ⬘共t兲 共17兲 is the entropy flow and the positive quantityS˙i共t兲 =
兺
m,m⬘, Wm,m共兲 ⬘共t兲pm⬘共t兲ln Wm,m共兲 ⬘共t兲pm⬘共t兲 Wm共兲⬘,m共t兲pm共t兲 =1 2兺
m,m⬘, Jm,m共兲⬘共t兲Xm,m共兲 ⬘共t兲 ⱖ 0 共18兲is identified as the EP. The latter is zero if and only if the condition of detailed balance is satisfied,
Wm,m共兲⬘共兲pm⬘= Wm⬘,m 共兲 共兲p
m. 共19兲
We note an important property of the EP. If all the relevant processescausing transitions between states m are not cor-rectly identified 共for example, if one only identifies a sub-class of these processes兲 the EP will be underestimated. In-deed, using the logarithmic-sum inequality 共cf. Theorem 2.7.1 of 关66兴兲, it follows that 共see also 关52–54兴兲
S˙i共t兲 =
兺
m,m⬘, Wm,m共兲 ⬘共t兲pm⬘共t兲ln Wm,m共兲 ⬘共t兲pm⬘共t兲 Wm共兲⬘,m共t兲pm共t兲 ⱖ兺
m,m⬘ Wm,m⬘共t兲pm⬘共t兲ln Wm,m⬘共t兲pm⬘共t兲 Wm⬘,m共t兲pm共t兲 ⱖ 0. 共20兲 B. Thermodynamic interpretationvariables共e.g., temperature or chemical potential兲. The tran-sitions between states m due to different mechanisms can, for example, correspond to exchange of heat with different reservoirs or the change in number of particles due to differ-ent chemical reactions. As a result the transition rates asso-ciated to a given mechanismneed to obey the condition of local detailed balance,
Wm,m共兲 ⬘共t兲pm⬘ eq 共t,兲 = Wm⬘,m 共兲 共 t兲pm eq共 t,兲, 共21兲 where the equilibrium distribution peq共t,兲 is the stationary distribution that would be reached by the system if only a single mechanismwere present and for the frozen value of the control variable=t. In case multiple mechanisms are present, each reservoir tries unsuccessfully to impose its equilibrium distribution on the system resulting in a station-ary distribution that does not satisfy the detailed balance con-dition 共19兲 except if their thermodynamical properties are identical, making the distinction between the various mecha-nisms useless. As we have seen in Eq. 共20兲, an incorrect identification of the various reservoirs would underestimate the EP and could lead one to believe that a system is at equilibrium while it is not关8兴. The assumption that the same expressions for the transition probabilities in Eq.共21兲 can be used when the control parameter becomes time dependent is based on an assumption that the idealized reservoirs relax infinitely fast to their equilibrium compared to the time scales of the system dynamics.
By an argument of physical consistency, it follows that S˙i共t兲 is also equal to the total EP S˙tot共t兲 in system plus ronment. Indeed, since one implicitly assumes that the envi-ronment remains at equilibrium at all times using Eq.共21兲, it does not have an internal EP of its own. Otherwise, the above description in terms of the system alone is not complete, and one needs to include the description of the irreversible pro-cesses taking place in the environment. For the same reason, the entropy flow is equal to minus the entropy increase in the reservoir S˙e共t兲=−S˙r共t兲. The microscopic origin of these rela-tions has been recently clarified关67兴.
In order to make the thermodynamic interpretation of the stochastic dynamics transparent, we now explicitly evaluate the various entropies for multiple heat reservoirs satisfying local detailed balance with respect to the canonical equilib-rium distribution at the inverse temperature of the corre-sponding reservoir, i.e., Eq.共21兲 becomes
Wm,m共兲 ⬘共兲
Wm共兲⬘,m共兲= exp兵− 共兲关⑀
m共兲 −⑀m⬘共兲兴其, 共22兲 where⑀m共兲 is the energy of the system when in the state m for the value of the control variable. We note that in order to make the connection between the system Shannon entropy 共10兲 and the true thermodynamic entropy, one might have to add a contribution兺mSmpm共t兲 to the entropy, where Smis the entropy of each level m关32,68,69兴. However, for simplicity, we now assume that the index m refers to the nondegenerate microscopic state of the system 共i.e., Sm= 0兲. It immediately follows from Eq.共22兲 that the entropy flow 共17兲 takes on the familiar thermodynamic form of heat flux over temperature,
S˙e共t兲 =
兺

共兲Q˙共兲共t兲, 共23兲
where the heat from the reservoir is given by Q˙共兲共t兲 =
兺
m,m⬘
Jm,m共兲 ⬘共t兲⑀m⬘共兲. 共24兲 Since the system energy is obviously given by
E共t兲 =
兺
m⑀m共兲pm共t兲, 共25兲
we find from the first principle of thermodynamics 共energy conservation兲,
E˙ 共t兲 = W˙共t兲 +
兺
Q˙共兲共t兲, 共26兲 that the work is given by
W˙ 共t兲 =
兺
m⑀˙m共兲pm. 共27兲
This illustrates that the local detailed balance conditions with the reservoirs 共22兲 lead to a proper formulation of the first 关Eq. 共26兲兴 and second 关Eq. 共16兲兴 principles of thermodynam-ics. We note that it is possible to include particle exchanges with the reservoirs关51兴. In the latter case, work can be non-zero even in the absence of driving.
We should finally mention that the system could also be driven out of equilibrium by a nonconservative force instead of different reservoirs. In this case, the local detailed balance condition共21兲 is not satisfied and even in the presence of a single reservoir, the detailed balance condition共19兲 will not be satisfied at steady state.
C. Adiabatic and nonadiabatic entropy balance We next note that the force X 共13兲 can be split in an adiabatic contribution A and a nonadiabatic contribution N,
Xm,m共兲 ⬘共t兲 = Am,m共兲 ⬘共t兲 + Nm,m⬘共t兲, 共28兲 Am,m共兲⬘共t兲 = ln Wm,m共兲⬘共t兲pm⬘ st 共t兲 Wm共兲⬘,m共t兲pm st共 t兲 , 共29兲 Nm,m⬘共t兲 = lnpm st共 t兲pm⬘共t兲 pmst⬘共t兲pm共t兲 . 共30兲
S˙na共t兲 = 1 2
兺
m,m⬘ Jm,m⬘共t兲Nm,m⬘共t兲 ⱖ 0 = −兺
m p˙m共t兲ln pm共t兲 pm st共 t兲 . 共33兲We make a number of remarks. First and most importantly, both S˙a共t兲 and S˙na共t兲 are non-negative EPs. This follows from Jensen’s inequality, −ln xⱖ1−x for x⬎0, together with Eqs. 共7兲 and 共9兲. The non-negativity of these quantities is in agreement with the fact that the trajectory entropies sa and sna obey detailed fluctuation theorems 关33兴. Second, it is clear that the nonadiabatic thermodynamic force, and hence the nonadiabatic EP, is zero in the adiabatic limit pm共t兲 →pm
st共
t兲. This will, for example, be the case when the re-laxation to the steady state is extremely fast, and in particular faster than the time scale of the drivingt. This observation justifies a posteriori the name given to each contribution. Third, we note that one does not need to identify the separate mechanisms by which the transition between states takes place for the evaluation of nonadiabatic EP or its correspond-ing fluxes and forces. It is a function only of the coarse-grained transition probabilities W =兺W. Finally, we note that for a system subjected to a nonconservative force and in contact with a single reservoir, the adiabatic EP is the house-keeping heat 共divided by the temperature of the reservoir兲 关22,60,63,70兴.
The fact that there are two contributions to the total en-tropy production, which are separately non-negative, sug-gests that the second law can be “split in two.” An elegant way to do so is by the introduction of the so-called excess entropy change关60,22兴, S˙ex共t兲 = 1 2
兺
m,m⬘ Jm,m⬘共t兲lnpm st共 t兲 pmst⬘共t兲 =兺
m p˙m共t兲ln pm st共 t兲. 共34兲 One easily verifies thatS˙共t兲 = − S˙ex共t兲 + S˙na共t兲, 共35兲 S˙r共t兲 = S˙ex共t兲 + S˙a共t兲. 共36兲 From these two relations, we see that the changes in system and reservoir entropy both have a structure similar to the original second law共16兲: they consist of the sum of a revers-ible and an irreversrevers-ible term if the concept of reversibility is understood with respect to the external driving in the first case共slow external driving implying S˙na= 0兲 and with respect to the nonequilibrium constraint exerted by the different res-ervoirs in the second case共for identical reservoirs S˙a= 0兲. As a result, the excess entropy change can be seen as minus the reversible change of the system entropy change with respect to the driving and as the reversible part of the reservoir en-tropy changes with respect to the nonequilibrium constraint. By summing these two relations we recover Eq.共31兲. Rela-tions共31兲, 共35兲, and 共36兲 thus represent the three faces of the second law.
III. SPECIFIC CLASS OF TRANSFORMATIONS
Additional comments can be made when considering the specific classes of transformation discussed below. To pro-ceed, it is useful to split the nonadiabatic entropy共33兲 into a “boundary” and a “driving” part关29兴,
S˙na共t兲 = S˙b共t兲 + S˙d共t兲, 共37兲 where S˙b共t兲 = − d dt
冉
兺
m pm共t兲ln pm共t兲 pm st共 t兲冊
, 共38兲 S˙d共t兲 = −兺
m pm共t兲 pm st共 t兲 p˙m st共 t兲. 共39兲The boundary contribution only depends on the initial and final distributions of the considered transformation while the driving part is only nonzero when the control variable evolves in time.
A. Transient relaxation to steady state
The system is supposed to be in an arbitrary state when the external driving is switched off, say at t = 0. For t⬎0, we have that t= is time independent, implying S˙d共t兲=0, and therefore
S˙na共t兲 = S˙b共t兲 = − H˙共t兲. 共40兲 Here, H共t兲 is the relative entropy 共or Kullback-Leibler en-tropy兲 关66兴 between the actual and the steady-state distribu-tions, H共t兲 = D„p共t兲储pst共兲… =
兺
m pm共t兲ln pm共t兲 pm st共兲ⱖ 0. 共41兲 We conclude that H共t兲 is a Lyapunov function, decreasing monotonically in time until the probability distribution reaches its steady-state value. This proof of convergence to the steady state is well known关71兴, but its connection to the nonadiabatic EP has never been pointed out. The nonadia-batic entropy assumes its minimum value equal to zero at the steady state, but the latter needs not be close to equilibrium.B. Transitions between steady states
We consider a system that is initially in a steady state pm共0兲=pm
st共
ti兲. Then somewhere between tiand tfthe driving variable t changes and after an asymptotically long time T the system has reached its new steady-state distribution pm共T兲=pm
st共
tf兲. Let us consider the total nonadiabatic en-tropy change during the time interval T. The boundary term is zero and the driving term is nonzero between ti and tf. Relation 共35兲 becomes the second law of steady-state ther-modynamics 关22,60兴,
C. Time-dependent cycles
We consider a system that is subjected to a time-periodic perturbation with period T. After an initial transient, the probability distribution and the various EP’s will also be-come time-periodic functions with the same period. Because of continuity, we have pm共0兲=pm共T兲. Furthermore, pm
st共
0兲 = pm
st共
T兲. Hence, both the boundary term along a cycle as well as the system entropy change over a period are zero, ⌬S共T兲=⌬Sb共T兲=0, and relation 共35兲 becomes
⌬Sna共T兲 = ⌬Sex共T兲 = ⌬Sd共T兲 ⱖ 0. 共43兲 We also have, using Eq. 共16兲,
⌬Stot共T兲 = ⌬Sr共T兲 = ⌬Sex共T兲 + ⌬Sa共T兲 ⱖ 0. 共44兲 D. Perturbing the steady state
Finally, we investigate how the adiabatic and nonadiabatic EPs change upon applying a perturbation around the steady-state distribution,
pm共t兲 = pm st共兲 +␦
pm共t兲. 共45兲
Clearly, flux共12兲 can be split as
Jm,m共兲 ⬘共t兲 = Jm,m共兲 ⬘共兲 +␦Jm,m共兲⬘共t兲, 共46兲 where J共兲m,m⬘共兲 = Wm,m共兲 ⬘共兲pstm⬘共兲 − Wm共兲⬘,m共兲pm st共兲, 共47兲 ␦Jm,m共兲 ⬘共t兲 = Wm,m共兲⬘共兲␦pm⬘共t兲 − Wm⬘,m 共兲 共兲␦ pm共t兲. 共48兲 The adiabatic contribution to the EP can be written as the sum of two contributions,
S˙a共t兲 = S˙a共兲 +␦S˙a共t兲 ⱖ 0, 共49兲 with S˙a共兲 = 1 2m,m
兺
⬘,Jm,m⬘ 共兲 共兲A m,m⬘ 共兲 共兲 ⱖ 0, 共50兲 ␦S˙a共t兲 = 1 2兺
m,m⬘, ␦Jm,m共兲 ⬘共t兲Am,m共兲⬘共兲. 共51兲 We now turn to the nonadiabatic EP which using Eqs.共33兲 and共45兲 now readsS˙na共t兲 = −
兺
m ␦p˙m共t兲ln pm共t兲 pm st共兲ⱖ 0. 共52兲 In analogy to the total EP关which can be written in a bilinear form in terms of flux and forces 共18兲兴, we can write the nonadiabatic EP as a bilinear form of the nonadiabatic flux and force asS˙na共t兲 = −
兺
m,m⬘␦Jm,m⬘Nm,m⬘ⱖ 0. 共53兲
We conclude that the adiabatic EP has a constant共zero-order兲 and first-order terms in the deviation␦p around the
station-ary state pm st
, while the nonadiabatic EP is of second order. In general, since␦S˙a共t兲 can be negative, we can find situations where S˙a共t兲 and even S˙i共t兲 are smaller than S˙a共兲. This is the well-known result that the total EP is not necessarily mum in nonequilibrium steady states. It is however a mini-mum if the steady states correspond to equilibrium since in this case Am,m共兲 ⬘共兲=0 关6,8兴.
IV. APPLICATIONS A. Two-level system
We consider a system with two levels, m = 1 , 2, with pmas the probability to find the system in level m. Due to conser-vation of total probability, the master equation 共6兲 can be reduced to a single differential equation for the probability p = p2= 1 − p1 to be in level 2,
p˙共t兲 = −␥共t兲p共t兲 + W21共t兲, 共54兲 with ␥共兲=W21共兲+W12共兲. The steady-state solution is pst共兲=W21共兲/␥共兲.
We first present the general solution for a periodic pertur-bation, with a period T, without specifying the form of the rates at this stage. We focus on the long-time regime, where all transients have disappeared, and the time behavior of p共t兲 itself is periodic with period T. We need to solve Eq. 共54兲 subject to periodic boundary conditions p共0兲=p共T兲. For sim-plicity, we focus on the case of a piecewise constant pertur-bation. Hence, the driving period consists of two regimes I and II, namely,t=I for 0ⱕt⬍tI, andt=IIfor tIⱕt⬍T. The solution of Eq. 共54兲 reads 关pI
st = pst共 I兲, pII st = pst共 II兲兴 0ⱕ t ⬍ tI: p共t兲 = 关p共T兲 − pI st兴e−␥It + pI st , 共55兲 tIⱕ t ⬍ T: p共t兲 = 关p共tI兲 − pII st兴e−␥II共t−tI兲 + pII st . 共56兲 Using the matching condition共continuity of p兲 at the transi-tions between regions I and II, we find that
p共tI兲 = pI st共e␥ItI− 1兲 − p II st共e−␥II共T−tI兲− 1兲 e␥ItI− e−␥II共T−tI兲 , 共57兲 p共T兲 =pI st共e−␥ItI− 1兲 − p II st共e␥II共T−tI兲− 1兲 e−␥ItI− e␥II共T−tI兲 . 共58兲 Equations共57兲 and 共58兲 with Eqs. 共55兲 and 共56兲 provide the exact and explicit long-time solution of Eq.共54兲 under piece-wise constant periodic driving.
We now turn to the entropies. We consider the case of two different reservoirs = L , R, with corresponding transition rates W =兺W共兲. Entropies共31兲–共33兲 become
S˙na= p˙共t兲ln
共1 − p兲pst
共1 − pst兲p. 共61兲 As an illustration, we consider two physical models that are described by the above two-level master equation. The first is a single-electron quantum level in contact with a left共L兲 or a right共R兲 electronic reservoir 共= L , R兲. The level can thus be empty, 1 − p, or filled, p, and transitions between these two states correspond to the exchange of an electron with one of the two reservoirs. The rates are given by the Fermi golden rule rates for each of the reservoirs关72兴,
W21共兲=⌫f共兲, W21共兲=⌫关1 − f共兲兴, 共62兲 where f共x兲=共exp兵x其+1兲−1 is the Fermi distribution and⌫ is the system-reservoir couplings chosen for simplicity identi-cal for the two reservoirs. Physiidenti-cally, the changes in the driv-ing parameter =共⑀−兲/T could result from changes in the energy of the level⑀or in the temperature Tand chemi-cal potential of the reservoir. The thermodynamics of this model has been discussed in Refs. 关73,74兴. The time dependence of the rates can result from either the external control of the energy of the level共e.g., with an electric field兲 or the control of the reservoir chemical potentials.
Our second model represents a two-level atom interacting with left 共L兲 and right 共R兲 reservoirs of thermal light. The rates describing the transitions between the levels are then given by
W21共兲=⌫n共兲, W21共兲=⌫关1 + n共兲兴, 共63兲 where n共x兲=共exp兵x其−1兲−1 is the Bose-Einstein distribution and =⑀/T. The time dependence of the rates can thus come either from an external control of the energy spacing between the levels⑀or from the control of the temperature of the reservoirs T.
The probability distribution as well as the various entro-pies around the cycle can be analytically evaluated. For the purpose of illustration, we reproduce the typical behavior in Figs.1and2.
B. Chemical reaction model Consider the following chemical reaction:
A共L兲 kM共L兲 kA共L兲 M kA共R兲 kM共R兲 A共R兲. 共64兲
Denoting by m the number of particles of species M and using= L , R, we find that the corresponding transition rates are given by
Wm−1,m共兲 = kM共兲m, Wm+1,m共兲 = kA共兲A共兲共t兲. 共65兲 The concentrations of A共L兲共t兲 and A共R兲共t兲 are externally con-trolled in a time-dependent manner. Rescaling time by kM共L兲 + kM共R兲, we can redefine the rates as
Wm−1,m共兲 =␣共兲m, Wm+1,m共兲 =t共兲, 共66兲 where we introduced ␣共兲= kM共兲 kM共L兲+ kM共R兲 , t共兲= kA共兲A共兲共t兲 kM共L兲+ kM共R兲 . 共67兲
The resulting master equation reads 共m=0,1,... with con-vention p−1= 0兲 关71兴 p˙m=
兺
兵␣ 共兲共m + 1兲p m+1+t共兲pm−1−共t共兲+␣共兲m兲pm其 =共m + 1兲pm+1+tpm−1−共t+ m兲pm. 共68兲 0.0 0.2 0.4 0.6 0.8 1.0 0.1 0.2 0.3 0.4A 0.0 0.2 0.4 0.6 0.8 1.0 0 1 2 3 4 5 t ΛRt ΛLt pt pstt 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 t B Sna Sa StotFIG. 1. 共Color online兲 共A兲 Actual probability distribution along the cycle and stationary solution corresponding to the instantaneous values of the driving displayed in the inset.共B兲 The total, adiabatic, and nonadiabatic entropy productions along the cycle. The Bose rates共63兲 are used with ⌫=0.5. Also, T=1 and tI= 0.4.
0.0 0.2 0.4 0.6 0.8 1.0 0.05 0.10 0.15 0.20 A 0.0 0.2 0.4 0.6 0.8 1.0 0 1 2 3 4 5 t ΛRt ΛLt pt pstt 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 t B Sna Sa Stot
From first to second line, we used兺t共兲=tand兺␣共兲= 1. Equation共68兲 can be most easily solved by switching to the generating function
F共s,t兲 =
兺
m=0⬁
smpm共t兲. 共69兲
Using Eqs.共68兲 and 共69兲, one finds
tF共s,t兲 = 共s − 1兲tF共s,t兲 − 共s − 1兲sF共s,t兲. 共70兲 For the purpose of illustration, we focus on a specific time-dependent solution of this equation, for which the explicit analytical expression can be obtained. Indeed, one readily verifies by inspection that
F共s,t兲 = em¯共t兲共s−1兲 共71兲 is an exact solution of Eq. 共70兲, propagating in time. As suggested by the notation, m¯共t兲 is the time-dependent aver-age
m¯ = m¯共t兲 =
兺
m=0⬁
mpm共t兲, 共72兲
which obeys the following equation:
m¯˙共t兲 = t− m¯共t兲. 共73兲 Its exact time-dependent solution is given by
m ¯共t兲 = e−共t−t0兲
再
m¯共t 0兲 +冕
t0 t de共−t0兲 冎
. 共74兲We conclude that the propagating Poissonian distribution,
pm共t兲 = m¯m m!e
−m¯
, 共75兲
with the average given by Eq. 共74兲, is an exact time-dependent solution of the master equation. The correspond-ing frozen steady states are
pm st共t兲 =关m¯ st兴m m! e −m¯st , 共76兲 with m¯st=t. 共77兲
We note, using Eqs.共66兲 and 共75兲, that Wm+1,m共兲 pm共t兲
Wm,m+1共兲 pm+1共t兲 = t
共兲
␣共兲m¯共t兲. 共78兲
We see that at steady state, the detailed balance condition is not in general satisfied. Equilibrium is only attained if t共兲 =␣共兲t. We thus have two mechanisms bringing the system out of equilibrium: the breaking of detailed balance by the steady state and the time-dependent driving fromt共兲.
Using these analytical results, we can now obtain explicit results for entropies 共31兲–共33兲,
S˙tot共t兲 =
兺
共␣ 共兲m¯ − t 共兲兲ln␣共兲m¯ t共兲 , 共79兲 S˙a共t兲 =兺
共␣ 共兲m¯ − t 共兲兲ln␣共兲m¯st t共兲 , 共80兲 S˙na共t兲 = − m¯˙ ln m¯ t =共m¯ −t兲ln m¯ t , 共81兲which are all positive. Not surprisingly, the nonadiabatic contribution vanishes in the adiabatic limit m¯ =t. On the other hand, the adiabatic contribution vanishes under the in-stantaneous detailed balance condition t共兲=␣共兲t 共remem-bering thatt= m¯st兲.
To compare the results with the two-level model, we again assume that the system is subjected to a periodic piece-wise constant driving of period T. One finds
0ⱕ t ⬍ tI: m¯共t兲 = e−t关m¯共T兲 + I共et− 1兲兴, tIⱕ t ⬍ T: m¯共t兲 = e−共t−t0兲关m¯共tI兲 + II共et−tI− 1兲兴, 共82兲 where m¯共tI兲 = I共e−tI− 1兲 + II共e−T− e−tI兲 e−T− 1 , 共83兲 m¯共T兲 =I共e tI− 1兲 + II共eT− etI兲 eT− 1 . 共84兲
The resulting behavior of the probability distribution and the various EPs around the cycle are shown in Fig. 3.
0 1 2 3 4 5 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 t ΛtR ΛtL A mt Λt 0 1 2 3 4 5 0 2 4 6 8 t B Sna Sa Stot
FIG. 3. 共Color online兲 共A兲 Actual average m¯共t兲 and stationary
V. CONCLUSIONS
In this paper, we started by showing how to identify en-tropy and enen-tropy production for a stochastic dynamics de-scribed by Markovian master equations with time-dependent rates. The key element is that the entropy production can be “split” into two parts, each satisfying a second-law-like rela-tion. By assuming that the rates satisfy a local detailed bal-ance condition, we have also shown that the dynamics pro-vides a nonequilibrium thermodynamics description of the system. The thermodynamic implications of this splitting
re-main to be properly understood. To progress in this direction we calculated the various entropies on different exactly solv-able models. In the companion paper关64兴, we proceed simi-larly for a stochastic dynamics described by a Fokker-Planck equation.
ACKNOWLEDGMENT
M.E. was supported by the Belgian Federal Government 共IAP project “NOSY”兲.
关1兴 P. Glansdorff and I. Prigogine, Thermodynamic Theory of
Structure, Stability and Fluctuations共Wiley, New York, 1971兲.
关2兴 S. R. de Groot and P. Mazur, Non-equilibrium
Thermodynam-ics共Dover, New York, 1984兲.
关3兴 G. Nicolis,Rep. Prog. Phys. 42, 225共1979兲.
关4兴 I. Prigogine, Thermodynamics of Irreversible Processes 共Wiley-Interscience, New York, 1961兲. 2nd Ed.
关5兴 D. Kondepudi and I. Prigogine, Modern Thermodynamics 共Wiley, New York, 1998兲.
关6兴 J. Schnakenberg,Rev. Mod. Phys. 48, 571共1976兲. 关7兴 K. Sekimoto,Prog. Theor. Phys. Suppl. 130, 17共1998兲. 关8兴 L. Jiu-Li, C. Van den Broeck, and G. Nicolis, Z. Phys. B:
Condens. Matter 56, 165共1984兲.
关9兴 D. J. Evans, E. G. D. Cohen, and G. P. Morriss,Phys. Rev. Lett. 71, 2401共1993兲.
关10兴 D. J. Evans and D. J. Searles,Phys. Rev. E 50, 1645共1994兲. 关11兴 G. Gallavotti and E. G. D. Cohen,Phys. Rev. Lett. 74, 2694
共1995兲.
关12兴 G. Gallavotti and E. G. D. Cohen, J. Stat. Phys. 80, 931 共1995兲.
关13兴 C. Jarzynski,Phys. Rev. Lett. 78, 2690共1997兲. 关14兴 C. Jarzynski,Phys. Rev. E 56, 5018共1997兲. 关15兴 J. Kurchan,J. Phys. A 31, 3719共1998兲. 关16兴 G. E. Crooks,J. Stat. Phys. 90, 1481共1998兲.
关17兴 J. L. Lebowitz and H. Spohn,J. Stat. Phys. 95, 333共1999兲. 关18兴 T. Hatano,Phys. Rev. E 60, R5017共1999兲.
关19兴 C. Maes,J. Stat. Phys. 95, 367共1999兲. 关20兴 G. E. Crooks,Phys. Rev. E 60, 2721共1999兲. 关21兴 G. E. Crooks,Phys. Rev. E 61, 2361共2000兲.
关22兴 T. Hatano and S. I. Sasa,Phys. Rev. Lett. 86, 3463共2001兲. 关23兴 C. Maes and K. Netocny,J. Stat. Phys. 110, 269共2003兲. 关24兴 C. Maes, Séminaire Poincaré 2, 29 共2003兲.
关25兴 U. Seifert,Phys. Rev. Lett. 95, 040602共2005兲.
关26兴 B. Cleuren, C. Van den Broeck, and R. Kawai,Phys. Rev. Lett.
96, 050601共2006兲.
关27兴 D. Andrieux and P. Gaspard,J. Stat. Phys. 127, 107共2007兲. 关28兴 D. Andrieux and P. Gaspard,J. Stat. Mech.: Theory Exp. 2007,
P02006.
关29兴 M. Esposito, U. Harbola, and S. Mukamel,Phys. Rev. E 76, 031132共2007兲.
关30兴 H. Ge,Phys. Rev. E 80, 021137共2009兲.
关31兴 H. Ge and H. Qian,Phys. Rev. E 81, 051133共2010兲. 关32兴 B. Gaveau, M. Moreau, and L. S. Schulman,Phys. Rev. E 79,
010102共R兲 共2009兲.
关33兴 M. Esposito and C. Van den Broeck, Phys. Rev. Lett. 104,
090601共2010兲.
关34兴 L. Onsager,Phys. Rev. 37, 405共1931兲. 关35兴 L. Onsager,Phys. Rev. 38, 2265共1931兲.
关36兴 L. Onsager and S. Machlup,Phys. Rev. 91, 1505共1953兲. 关37兴 S. Machlup and L. Onsager,Phys. Rev. 91, 1512共1953兲. 关38兴 H. B. Callen and T. A. Welton,Phys. Rev. 83, 34共1951兲. 关39兴 R. F. Fox and G. E. Uhlenbeck,Phys. Fluids 13, 1893共1970兲. 关40兴 R. F. Fox and G. E. Uhlenbeck,Phys. Fluids 13, 2881共1970兲. 关41兴 R. Kubo,J. Phys. Soc. Jpn. 12, 570共1957兲.
关42兴 R. Kubo and S. Nakajima,J. Phys. Soc. Jpn. 12, 1203共1957兲. 关43兴 C. Van den Broeck, Selforganization by Nonlinear Irreversible
Processes共Springer, New York, 1986兲, p. 57.
关44兴 U. Seifert,Eur. Phys. J. B 64, 423共2008兲.
关45兴 M. J. Klein and P. H. E. Meijer,Phys. Rev. 96, 250共1954兲. 关46兴 C. Maes and K. Netocny,J. Math. Phys. 48, 053306共2007兲. 关47兴 S. Bruers, C. Maes, and K. Netocny, J. Stat. Phys. 129, 725
共2007兲.
关48兴 J. M. R. Parrondo and P. Espanol, Am. J. Phys. 64, 1125 共1996兲.
关49兴 D. Andrieux and P. Gaspard,J. Chem. Phys. 121, 6167共2004兲. 关50兴 C. Van den Broeck,Phys. Rev. Lett. 95, 190602共2005兲. 关51兴 M. Esposito, K. Lindenberg, and C. Van den Broeck, Phys.
Rev. Lett. 102, 130602共2009兲.
关52兴 R. Kawai, J. M. R. Parrondo, and C. Van den Broeck, Phys. Rev. Lett. 98, 080602共2007兲.
关53兴 A. Gomez-Marin, J. M. R. Parrondo, and C. Van den Broeck, EPL 82, 50002共2008兲.
关54兴 J. M. R. Parrondo, C. Van den Broeck, and R. Kawai,New J. Phys. 11, 073008共2009兲.
关55兴 J. Farago,J. Stat. Phys. 107, 781共2002兲. 关56兴 J. Farago,Physica A 331, 69共2004兲.
关57兴 R. van Zon and E. G. D. Cohen,Phys. Rev. Lett. 91, 110601 共2003兲.
关58兴 R. van Zon and E. G. D. Cohen, Phys. Rev. E 69, 056121 共2004兲.
关59兴 M. Baiesi, T. Jacobs, C. Maes, and N. S. Skantzos,Phys. Rev. E 74, 021111共2006兲.
关60兴 Y. Oono and M. Paniconi,Prog. Theor. Phys. Suppl. 130, 29 共1998兲.
关61兴 T. Speck and U. Seifert,EPL 74, 391共2006兲.
关62兴 V. Y. Chernyak, M. Chertkov, and C. Jarzynski,J. Stat. Mech.: Theory Exp. 2006, P08001.
关63兴 R. J. Harris and G. M. Schutz, J. Stat. Mech.: Theory Exp. 2007, P07020.
Rev. E 82, 011144共2010兲.
关65兴 P. Gaspard,J. Stat. Phys. 117, 599共2004兲.
关66兴 T. M. Cover and J. A. Thomas, Elements of Information
Theory共Wiley, New York, 2006兲.
关67兴 M. Esposito, K. Lindenberg, and C. Van den Broeck,New J. Phys. 12, 013013共2010兲.
关68兴 D. Andrieux and P. Gaspard, Proc. Natl. Acad. Sci. U.S.A.
105, 9516共2008兲.
关69兴 M. Esposito, K. Lindenberg, and C. Van den Broeck,J. Stat. Mech.: Theory Exp. 2010, P01008.
关70兴 T. Speck and U. Seifert,J. Phys. A 38, L581共2005兲. 关71兴 N. G. van Kampen, Stochastic Processes in Physics and
Chemistry, 2nd ed.共North-Holland, Amsterdam, 1997兲.
关72兴 U. Harbola, M. Esposito, and S. Mukamel,Phys. Rev. B 74, 235309共2006兲.
关73兴 M. Esposito, K. Lindenberg, and C. Van den Broeck,EPL 85, 60010共2009兲.
