On reversible information hiding system

(1)

Proceedings Chapter

Reference

On reversible information hiding system

HAROUTUNIAN, Mariam, et al.

Abstract

In this paper we consider the problem of reversible information hiding in the case when the attacker uses only discrete memoryless channels (DMC), the decoder knows only the class of channels, but not the DMC chosen by the attacker, the attacker knows the information-hiding strategy, probability distributions of all random variables, but not the side information. We introduce the notion of reversible information hiding Incapacity, which expresses the dependence of the information hiding rate on the error probability exponent E and the distortion levels for the information hider, for the attacker and for the host data approximation.

The random coding bound for reversible information hiding B-capacity is found. We obtain the lower bound for reversibility information hiding capacity for E rarr 0. In particular, we have analyzed two special cases of the general problem formulation, pure reversibility and pure message communications.

HAROUTUNIAN, Mariam, et al . On reversible information hiding system. In: Proceedings of the IEEE International Symposium on Information Theory, ISIT 2008 . IEEE, 2008. p.

940-944

DOI : 10.1109/ISIT.2008.4595125

Available at:

http://archive-ouverte.unige.ch/unige:47669

Disclaimer: layout of this document may differ from the published version.

1 / 1

(2)

On Reversible Information Hiding System

Mariam Haroutunian IIAP of NAS RA Yerevan, Armenia Email: [email protected]

Smbat Tonoyan IIAPof NAS RA Yerevan, Armenia Email: [email protected]

Oleksiy Koval University of Geneva

Geneva, Switzerland Email: [email protected]

Svyatoslav Voloshynovskiy University of Geneva

Geneva, Switzerland Email: [email protected]

Abstract—In this paper we consider the problem of reversible information hiding in the case when the attacker uses only discrete memoryless channels (DMC), the decoder knows only the class of channels, but not the DMC chosen by the attacker, the attacker knows the information-hiding strategy, probability distributions of all random variables, but not the side informa- tion.

We introduce the notion of reversible information hiding E- capacity, which expresses the dependence of the information hiding rate on the error probability exponentEand the distortion levels for the information hider, for the attacker and for the host data approximation. The random coding bound for reversible information hiding E-capacity is found. We obtain the lower bound for reversibility information hiding capacity for E→0.

In particular, we have analyzed two special cases of the general problem formulation, pure reversibility and pure message communications.

I. INTRODUCTION

Problem of information transmission over state dependent channels rises in the situation when the transmitter has a certain prior knowledge about the environment or channel.

Such a situation is typical in data hiding where the main goal is to communicate information message embedded into the body of a media file over a certain channel [1]. It is also relevant to simultaneous transmission of digital and analog information in audio broadcasting [2].

A corresponding research area of communications over channels with side information available at the transmitter has attracted considerable attention in information theory. The main problem was to establish the highest possible rates of reliable communications in such channels [3]-[6] that is related to the optimal solution to the problem of channel interference cancellation.

However, in certain cases one is rather interested not in maximizing the rate of pure information transmission [7] but in the most accurate estimate of the channel interference (channel state). Such a problem arises in simultaneous broadcasting of analog and digital audio [2] where digital data designed in a way to enhance the overall reception quality can be considered as interference degrading the communication of analog information.

The problem of accurate channel state estimation at the output of the state-dependent channel (reversibility) was studied in communication and data hiding formulation. Sutivong et al. [7] considered the problem of simultaneous channel state transmission in addition to the pure information. They present the optimal protocol design and rate distortion region for pure

information transmission / partial channel state recovery for the state-dependent channel. Similar results were obtained in [8] for multimedia authentication formulation of the problem.

The maximum achievable rates for pure information transmission in the case of complete recovery of the channel state at the decoder are analyzed in [9]. One should also mention the work of Eggers et al. [10], who considered the reversibility of quantization-based data hiding as structured codebook approximation of random binning. Finally, the analysis of [11]

considers a formulation where the communication protocol is specifically optimized to a particular pure information communication regime while reversibility is analyzed as a by- product of this design. Similarly to the previous cases, the rate- distortion region of pure information transmission and channel state recovery is defined.

In this paper we would like to make one step forward with respect to the existing results justifying joint information transmission rates and channel state estimation accuracy at the output of the data hiding channel and to establish the error exponents that can be attained in the reversible information hiding protocols in terms of E-capacity [12]-[14].

II. PROBLEMFORMULATION

We use capital letters X to denote random variables (RV) and corresponding small lettersxfor their realizations. Small bold lettersxdesignate length-N vectorsx= [x1, x2, ..., xN] withn^thelementxn. Calligraphic fontsX denote sets and|X | denotes the cardinality of setX. All logarithms and exponents in the paper are of the base2. We use the following notation m = 1,|M| for m = 1,2, . . . ,|M|. Information-theoretic quantities, such as conditional entropy of RV Y relative to RV X with probability density (PD) P0,V ={V(y|x), x ∈ X, y ∈ Y} will be denoted as HP0,V(Y|X); the conditional mutual information of the RVSandSˆrelative to RV byK is IQ,Q2(S∧S|K); the informational divergence of the PDˆ Q^∗ andQis denoted by D(QkQ^∗) and the conditional informational divergence of joint PDQ^∗◦Q2◦P◦V andQ^∗◦Q2◦P◦A byD(Q^∗◦Q2◦P◦VkQ^∗◦Q2◦P◦A) =D(VkA|Q^∗, Q2, P).

We denote the types or empirical distributions by small letters.

The set of all vectorskof typeq0we denote byT_q^N₀(K). The set of all vectors s ∈ S^N of conditional type q1 for given k∈ T_q^N₀(K)we denote byT_q^N₁(S|k). It is called alsoq-shell of vectork. The notation |a|⁺ will be used formax(a,0).

(3)

- - - -

66 6

m

s k

Encoder fN

x y Decoder m^′ Receiver

gN M

M S^N K^N

Attack Channel A(y|x) Message

source Host data

source informationSide

source

Figure 1. Reversible information hiding system

-

ˆs Sˆ^N ......

......

...

......

...

A reversible information hiding system is presented in Figure 1. It is supposed that a messagemto be transmitted to the receiver is uniformly distributed over the message setM.

Host data source is described by the RVS, which takes values in the discrete finite set S and generates N-length sequences of independent and identically distributed (i.i.d.) components.

The side information source is described by the RVK, which takes values in the discrete finite setK, and in the most general case has the given joint PDQ^∗={Q^∗(s, k), s∈ S, k∈ K}

with the RV S. When the side information is a cryptographic key, S and K are independent. The side information in the form of i.i.d. N-length sequences is available to the encoder and decoder. It is assumed that Q^∗N(s,k) =

N

Q

n=1

Q^∗(s, k).

The information hider (encoder) embeds the message m∈ Min the host data blockss∈ S^N using the side information k ∈ K^N. The resulting codeword x ∈ X^N is transmitted via attack channel A = {A(y|x), x ∈ X, y ∈ Y} with the finite input and output alphabets X and Y. The attacker, trying to modify or remove the messagem, produces corrupted blocksy∈ Y^N based onx∈ X^N, respectively. The decoder, possessing side information, derives the message m^′ and the approximation ˆs of the original data block, within the fixed distortion level usingy.

Let the mappings d0 : S ×S →ˆ [0,∞), d1 : S × X → [0,∞),d2:X × Y →[0,∞), be single-letter distortion functions. The distortion functions are supposed to be symmetric:

d0(s,s) =ˆ d0(ˆs, s), d1(s, x) = d1(x, s), d2(x, y) = d2(y, x) for all s ∈ S,sˆ ∈ S, xˆ ∈ X, y ∈ Y and assume that d0(s,s) = 0,ˆ if s = ˆs, d1(s, x) = 0, if s= x, d2(x, y) = 0, if x = y. Distortion functions for the N-length vectors are defined as d^N₀(s,ˆs) = _N¹

N

P

n=1

d0(sn,sˆn), d^N₁(s,x) =

1 N

N

P

n=1

d1(sn, xn), d^N₂ (x,y) = _N¹ P^N

n=1

d2(xn, yn).

Definition 1. The information hidingN-length code is a pair of mappings (fN, gN) subject to distortions ∆0,∆1, where fN : M × S^N × K^N → X^N is the encoder, mapping host data block s, the message m and side information k to x = fN(s, m,k), which satisfies the following distortion constraint:

d^N₁(s, fN(s, m,k))≤∆1, (1) andgN : Y^N×K^N → M×Sˆ^N is the decoding, mapping the received sequence y and side information k to the decoded message m^′ and ˆs, which satisfies the following distortion constraint: d^N₀(s,ˆs)≤∆0.

Note that the definition of the distortion constraint (1) means that the maximum distortion constraint with respect to s,k and m is used, as distinct from [17], where the average

distortion constraint is considered, and the maximum distortion constraint is mentioned as a more difficult case.

The attack channel, defined by A^N(y|x) =

N

Q

n=1

A(yn|xn), subject to distortion∆2, satisfies the following constraint:

X

x∈X^N

X

y∈Y^N

d^N₂ (x,y)A^N(y|x)p^N(x)≤∆2.

Definition 2. The nonnegative number R = _N¹ log|M| is called the information hiding code rate.

For any Q = Q0◦Q1 = {Q(s, k) = Q0(k)Q1(s|k), s ∈ S, k ∈ K} and ∆0, denote by Q2(Q,∆0) the set of all conditional PDsQ2(ˆs|s, k), for which the following inequality takes place:

X

s,ˆs,k

Q(s, k)Q2(ˆs|s, k)d0(s,s)ˆ ≤∆0. (2)

Definition 3. A memoryless covert channel P, subject to distortion ∆1, is a PD P = P0◦P1 = {P(u, x|s,s, k) =ˆ P0(u|s,s, k)Pˆ 1(x|u, s,ˆs, k), u ∈ U, x ∈ X, s ∈ S,ˆs ∈ S, kˆ ∈ K} such that for anyQandQ2∈ Q2(Q,∆0):

X

u,x,s,ˆs,k

Q(s, k)Q2(ˆs|s, k)P(u, x|s,s, k)dˆ 1(s, x)≤∆1, (3)

whereU is an auxiliary RV taking values in the finite setU and forming the following Markov chain(K, S,S, Uˆ )→X →Y. Denote by P(Q, Q2,∆1) the set of all covert channels P^N(u,x|s,ˆs,k) = Q^N

n=1

P(un, xn|sn,sˆn, kn), subject to distortion∆1.

Definition 4. A memoryless attack channel A, subject to distortion ∆2, under the condition of covert channel P ∈ P(Q, Q2,∆1), is defined by a PD A, for which for Q and Q2∈ Q2(Q,∆0)

X

u,x,s,ˆs,k,y

Q(s, k)Q2(ˆs|s, k)P(u, x|s,s, k)A(y|x)dˆ 2(x, y)≤∆2.

Denote by A(Q, Q2, P,∆2) the set of all attack channels, under the condition of covert channel P ∈ P(Q, Q2,∆1) and subject to distortion level ∆2. The sets Q2(Q,∆0),P(Q, Q2,∆1) and A(Q, Q2, P,∆2) are defined by linear inequality constraints and hence are convex.

Denote byg_N,k⁻¹(m,ˆs)the set of allywhich for a givenkare decoded into(m,ˆs): g⁻¹_N,k(m,ˆs) ={y: gN(y,k) = (m,ˆs)}.

Definition 5. The probability of erroneous reconstruction of the message m ∈ M and the approximation of data block s∈ S^N fork∈ K^N obtained at the output of the channelA is:

e(m,s,k, A) = 1−A^N





 [

ˆs:d(s,ˆs)≤∆0

g⁻¹_N,k(m,ˆs)|fN(m,s,k)





 .

The error probability of the message m averaged over all (s,k)∈ S^N× K^N equals to:

e(m, A) = X

(s,k)∈ S^N×K^N

Q^∗N(s,k)e(m,s,k).

(4)

Denote by ∆ = [∆0,∆1,∆2] the collection of distortion levels, fixed for the current system.

The error probability of the code, for any message m ∈ M, maximal over all attack channels from A(Q, Q2, P,∆2) is denoted by:

e(m) = max

A∈A(Q,Q2,P,∆2)e(m, A).

The maximal error probability of the code over all attack channels from A(Q, Q2, P,∆2) is equal to:e= max

m∈Me(m), and the average error probability of the code, maximal over all attack channels from A(Q, Q2, P,∆2) equals to: e =

1

|M|

P

m∈M

e(m).

III. REVERSIBLEINFORMATIONHIDINGE-CAPACITY

Consider the codes whose maximal error probability exponentially decreases with the given exponent E > 0, (called reliability), i.e., e≤exp{−N E}.

Denote byM(Q^∗, E, N,∆)the highest volume of the code, satisfying this condition for the given reliability E and the distortion levels∆.

The rate-reliability-distortion function, which we call re- versible information hiding E-capacity by analogy with the E-capacity of ordinary channel [12], is defined as:

R(Q^∗, E,∆) =C(Q^∗, E,∆)= lim^△

N→∞

1

N logM(Q^∗, E, N,∆).

ByC(Q^∗, E,∆)andC(Q^∗, E,∆)we denote the reversible information hidingE-capacity defined for maximal and average error probabilities respectively.

In this paper the lower bound of reversible information hidingE-capacity for maximal and average error probabilities is constructed.

Consider the following function, which we call the random coding bound

Rr(Q^∗, E,∆) = min

Q max

Q2∈Q2(Q,∆0) max

P∈P(Q,Q2,∆1)

min

A∈A(Q,Q2,P,∆2) min

V:D(Q◦Q2◦P◦VkQ^∗◦Q2◦P◦A)≤E

IQ,Q2,P,V(Y ∧U|K)−IQ,Q2,P0(S∧U,S|K)ˆ

+D(Q◦Q2◦P◦VkQ^∗◦Q2◦P◦A)−E|⁺. (4)

Theorem. For any E > 0, for reversible information hiding system with distortion levels ∆

Rr(Q^∗, E,∆)≤C(Q^∗, E,∆)≤C(Q^∗, E,∆).

Corollary 1. When E → 0, we obtain the lower bound of reversible information hiding capacity based on (4) :

Rr(Q^∗, E,∆) = max

Q2∈Q2(Q^∗,∆0) max

P∈P(Q^∗,Q2,∆1) min

A∈A(Q^∗,Q2,P,∆2)

IQ^∗,Q2,P,A(Y ∧U|K)−IQ^∗,Q2,P0(S∧U,Sˆ|K) .

Corollary 2: pure reversibility. If∆0= 0from (4) we have Rr(Q^∗, E,∆) = min

Q max

P∈P(Q,∆1) min

A∈A(Q,P,∆2)

V:D(Q◦P◦VminkQ^∗◦P◦A)≤E|IQ,P,V(Y ∧U|K)

−HQ(S|K) +D(Q◦P◦VkQ^∗◦P◦A)−E|⁺. (5)

Corollary 3: pure message communications. If ∆0 → ∞ then

Rr(Q^∗, E,∆) = min

Q max

P∈P(Q,∆1) min

A∈A(Q,P,∆2)

min

V:D(Q◦P◦VkQ^∗◦P◦A)≤E|IQ,P,V(Y ∧U|K)

−IQ,P(S∧U|K) +D(Q◦P◦VkQ^∗◦P◦A)−E|⁺. (6)

In (5) and (6), P = {P(u, x|s, k), u ∈ U, x ∈ X, s ∈ S, k∈ K} and∆= (∆1,∆2).

IV. PROOF OF THETHEOREM

The theorem is proved using Shannon’s random coding argument, the method of types, covering lemma and a gen- eralization of packing lemma [12], [15], [16].

To prove the random coding bound, we must show the existence of a code withRsatisfying (4) ande≤exp{−N(E− ε)}, for any0< ε < E.

We will construct the encoding and the decoding and explore the errors caused by each.

For encoding we use the idea of Gelfand-Pinsker [5].

The decoding is based on minimum divergence method, first introduced by E. Haroutunian [12] and developed in [13], [14]. The extension of this method adopted to data hiding can be considered as semi-universal decoding, since the decoder needs to know only the specific class of channels, instead of a particular one, used by the attacker.

Encoding.

Step 1. Denote byQ(Q^∗, E) ={q: D(qkQ^∗)≤E} and T_Q^E∗(S, K) = [

q∈Q(Q^∗,E)

T_q^N(S, K). (7)

We will construct the code only for(s,k)fromT_Q^E∗(S, K), because for sufficiently large N, the probability of (s,k) ∈/ T_Q^E∗(S, K)is exponentially small:

Q^∗N





 [

q /∈Q(Q^∗,E)

T_q^N(S, K)







= X

q /∈Q(Q^∗,E)

Q^∗N{T_q^N(S, K)}

≤ X

q /∈Q(Q^∗,E)

exp{−N D(qkQ^∗)}

<(N+ 1)^|S||K|exp{−N E} ≤exp{−N(E−ε1)}, (8) where ε1>0.

Step 2. Denote qˆ1(ˆs|k) =P

s

q2(ˆs|s, k)q1(s|k).

Covering lemma. For every type q, conditional type q2, vector k ∈ K^N, there exists a collection of vectors {ˆs_j ∈ T_q_ˆ^N₁( ˆS|k), j= 1, J1}, where

J1= exp

N Iq,q2(S∧S|K) +ˆ δ/2 , δ >0

(5)

such that the set

T_q,q^N₂(S|ˆsj,k) j= 1, J1 coversT_q^N₁(S|k)forN large enough:

T_q^N₁(S|k)⊂

J

[

j=1

T_q,q^N₂(S|ˆsj,k).

For the proof of covering lemma see [15].

For type q ∈ Q(Q^∗, E) and conditional type q2 ∈ Q2(q,∆0)denote

S(q, q2, j) =T_q,q^N₂(S|ˆs_j,k)\ [

j^′<j

T_q,q^N₂(S|ˆs_j^′,k),

therefore for the vectors s from S(q, q2, j), we put into correspondence the vectorˆs_j, j∈[1, J1].

Taking into account the inequality (2), we can show that for types q ∈ Q(Q^∗, E), q2 ∈ Q2(q,∆0) and any j = 1, J1, s∈ S(q, q2, j),ˆs_j

d0(s,ˆsj) =N⁻¹X

s,ˆs

n(s,ˆs|s,ˆsj)d0(s,s)ˆ

=X

s,ˆs,k

q(s, k)q2(ˆs|s, k)d0(s,ˆs)≤∆0, j= 1, J1.

Step 3. Fix the type p=p0◦p1 ∈ P(q, q2,∆1). For fixed p0, E, for each typeq∈ Q(Q^∗, E),q2∈ Q(q,∆0)and vectors ˆ

s,k, we choose independently, at random fromT_q^N₀_,p₀(U|ˆs,k)

|M| collections J2(m), m= 1,|M|, of vectors u_j(m), j= 1, J2, where

J2= exp

N Iq,q2,p0(S∧U|S, K) +ˆ δ/2 (δ >0).

Then, for each s ∈ T_q^N(S|k) we select such u_j(m) from J2(m), that u_j(m) ∈ T_q,q^N₂_,p₀(U|s,ˆs,k). Denote this vector by u(m,s,ˆs,k).

If for somesthere is no such a vector inJ2(m), we choose u(m,s,ˆs,k) at random from the T_q,q^N₂_,p₀(U|s,ˆs,k). Denote the probability of such an event by Pr{bq,q2,p0(m,s,ˆs,k)}.

Pr{bq,q2,p0(m,s,ˆs,k)}

= Pr (_J₂

\

j=1

u_j(m)∈ T/ q,q^N2,p0(U|s,ˆs,k) )

≤

J2

Y

j=1

[1−Pr{uj(m)∈ T_q,q^N₂_,p₀(U|s,ˆs,k)}]

≤

1−|Tq,q^N2,p0(U|s,ˆs,k)|

|Tq^N0,p0(U|ˆs,k)|

J2

≤[1−exp{−N(Iq,q2,p0(S∧U|S, K)ˆ +δ/4)}]^exp{N(I^q,q²^,p⁰^(S∧U|^S,K)+δ/2)}^ˆ .

Using the inequality(1−t)ⁿ≤exp{−nt}, which holds for any nandt∈(0,1), we can see that

Pr{bq,q2,p0(m,s,ˆs,k)} ≤exp{−exp{N δ/4}}. (9) Notice that for each m, the code contains J =J1×J2= expn

N

Iq,q2,p0(S∧U,S|K) +ˆ δo

(δ >0) vectorsu.

Step 4. The codewordxis constructed in the following way.

For each m = 1,|M|, ˆs, s and k we choose at random a vectorx(m,s,ˆs,k)fromT_q,q^N₂_,p(X|u(m,s,ˆs_j,k),s,ˆs,k).

It is easy to demonstrate that such an encoding satisfies the distortion constraint. Indeed, for types p∈ P(q, q2,∆1),q ∈ Q(Q^∗, E), q2∈ Q2(q,∆0), taking into account the inequality (3), we have

d^N1 (s,x(m,s,ˆs,k)) =N⁻¹X

s,x

n(s, x|s,x)d1(s, x)

= X

u,x,s,ˆs,k

q(s, k)q2(ˆs|s, k)p(u, x|s,s, k)dˆ 1(s, x)≤∆1.

Denote by eE(m) the encoding error probability for any m∈ M:

eE(m)≤ X

(s,k)∈T/ ^E Q∗(S,K)

Q^∗N(s,k)

+ X

(s,k)∈T_Q^E_∗(S,K)

Q^∗N(s,k) Pr{bq,q2,p0(m,s,ˆs,k)}.

Now taking into account (7), (8) and (9):

eE(m)≤exp{−N(E−ε1)}

+ X

q∈Q(Q^∗,E)

Q^∗N

Tq^N(S, K) exp{−exp{N δ/4}}.

As the number of types q in Q(Q^∗, E) does not exceed (N + 1)^|S||K| according to type counting lemma [15], [16]

andQ^∗N

T_q^N(S, K) ≤1, we can write

eE(m)≤exp{−N(E−ε1)}+ exp{−exp{N δ/4}+ε1}, (10) for N large enough.

The attacker chooses the attack channel A from the set A(q, q2, p,∆2) as he knows probability distributions of all random variables. It is clear, that in this case the average distortion constraint is satisfied, since:

X

x∈X^N

X

y∈Y^N

d^N₂(x,y)A^N(y|x)p^N(x)

=Ed^N₂(X^N, Y^N) = 1 N

N

X

n=1

Ed2(xn, yn)

= X

u,x,s,ˆs,k,y

q(s, k)q2(ˆs|s, k)p(u, x|s,s, k)A(y|x)dˆ 2(x, y)≤∆2.

Decoding. For brevity the vector pair u(m,s,ˆs,k),x(m,s,ˆs,k)is denoted byu,x(m,s,ˆs,k).

We use the following decoding rule: every pair of y and k is decoded to such m and ˆs that y ∈ T_q,q^N₂_,p,v(Y|u,x(m,s,ˆs,k),s,ˆs,k), where q, q2, p, v are such

that min

A∈A(q,q2,p,∆2)D(q◦q2◦p◦vkQ^∗◦q2◦p◦A)is minimal.

The decoder can make an error when the messagem∈ M is transmitted and (s,k) ∈ T_Q^E∗(S, K). However, there exist such typesq^′, q₂^′, p^′, v^′, vectors^′and pair(m^′,ˆs^′)thatm^′ 6=m or m^′=m,d0(s,ˆs^′)>∆0, with

y∈ T_q,q^N₂_,p,v(Y|u,x(m,s,ˆs,k),s,ˆs,k)

(6)

\T_q′,q^′₂,p^′,v^′(Y|u^′,x^′(m^′,s^′,ˆs^′,k),s^′,ˆs^′,k)

and min

A∈A(q^′,q₂^′,p^′,∆2)D(q^′◦q₂^′ ◦p^′◦v^′kQ^∗◦q₂^′ ◦p^′◦A)

≤ min

A∈A(q,q2,p,∆2)D(q◦q2◦p◦vkQ^∗◦q2◦p◦A). (11) Denote byD={q, q^′, p, p^′, q2, q₂^′, v, v^′:(11) is valid}and

F =T_q,q^N₂_,p,v(Y|u,x(m,s,ˆs,k),s,ˆs,k)

\ [

(m^′,ˆs^′): m′ 6=m or m′=m, d0(s,ˆs′)>∆0

[

s^′∈T^N q′,q′

2 (S|ˆs^′,k)

T_q^N′,q₂^′,p^′,v^′(Y|u^′,x^′(m^′,s^′,ˆs^′,k),s^′,ˆs^′,k).

The decoding error probabilityeD(m)of messagem∈ M, maximal over all attack channels A∈ A(q, q2, p,∆2), can be estimated in the following way:

eD(m)≤ max

A∈A(q,q2,p,∆2)

X

(s,k)∈T_Q^E_∗(S,K)

Q^∗N(s,k)

×A^N (

[

D

F|x(m,s,ˆs,k) )

≤X

D

|F|

× max

A∈A(q,q2,p,∆2)

X

(s,k)∈T_Q^E_∗(S,K)

Q^∗N(s,k)A^N(y|x).

The last inequality is true, because for fixed types of xand y the probabilityA^N(y|x)is constant.

For the estimation of decoding error probability we use the statement of the following lemma, which is the modification of packing lemma from [14].

Packing Lemma. For any E >2δ≥0, fixedq∈ Q(Q^∗, E), q2 ∈ Q2(q,∆0) and covert channel p ∈ P(q, q2,∆1), there exists a code with

|M|= exp

N min

A∈A(q,q2,p,∆2) min

v:D(q◦q2◦p◦vkQ^∗◦q2◦p◦A)≤E

|Iq,q2,p,v(Y ∧U|K)−Iq,q2,p0(S∧U,S|K)ˆ +D(q◦q2◦p◦vkQ^∗◦q2◦p◦A)−E−2δ|⁺ , such that

1) for eachk,sandˆs, the vector pairsu,x(m,s,ˆs,k)are distinct for differentm∈ M,

2) for sufficiently large N the following inequality holds for any types q^′ ∈ Q(Q^∗, E), q^′₂ ∈ Q2(q^′,∆0), p^′ ∈ P(q^′, q^′₂,∆1), v, v^′, and for all m = 1,|M|, (s,k) ∈ T_q^N(S, K),ˆs∈ T_q^N₂( ˆS|s,k)

|F| ≤ |T_q,q^N₂_,p,v(Y|u,x(m,s,ˆs,k),s,ˆs,k)|×exp{−N|E

− min

A∈A(q^′,q₂^′,p^′,∆2)D(q^′◦q₂^′ ◦p^′◦v^′kQ^∗◦q^′₂◦p^′◦A)

+) . (12) The lemma guarantees the existence of a good code, the codewords of which must be far from each other in a sense that allq, v-shells have possibly small intersections.

Using (11) and (12) it is easy to see that forNlarge enough eD(m)≤exp{−N(E−ε2)}, (13)

where ε2>0.

From (10) and (13) one can see that the error probability of the messagem∈ M is small enough.

Taking into account the continuity of all expressions, when N → ∞, arbitrary probability distributions can be considered instead of types. The theorem is proved.

V. ACKNOWLEDGMENTS

This paper was partially supported by SNF PB grant 114613 and SNF 200021–111643, by the EU projects IST-2002- 507932-ECRYPT, FP6-507609-SIMILAR and Swiss IM2 project.

REFERENCES

[1] I. J. Cox, M. L. Miller, J.A. Bloom, ”Digital Watermarking”, Morgan Kaufmann Publishers, Inc., San Francisco, 2001.

[2] H. C. Papadopoulos and C.-E. W. Sundberg, ”Simultaneous broadcasting of analog FM and digital audio signals by means of adaptive precanceling techniques”, IEEE Trans. on Communicationsy, vol. 46, num. 9, p. 1233-1242, 1998.

[3] C. E. Shannon, ”Channels with side information at the transmitter”, IBM J. Res. Develop, vol. 2, pp. 289–293, 1958.

[4] A. V. Kusnetsov and B. S. Tsybakov, ”Coding in a memory with defective cells”, translated from Prob. Peredach. Inform., vol. 10, no.

2, pp. 52–60, April-June 1974.

[5] S. I. Gel’fand and M.S. Pinsker, ”Coding for channel with random parameters”, Prob. Cont. and Inf. Theory, vol. 9, no. 1, p. 19-31, 1980.

[6] C. Heegard and A. El Gamal, ”On the capacity of computer memories with defects”, IEEE Trans. Inform. Theory, vol. 29, pp. 731–739, September 1983.

[7] A. Sutivong, M. Chiang, T.M. Cover, and Y.-H. Kim, ”Channel Ca- pacity and State Estimation for State-Dependent Gaussian Channels”, IEEE Trans. on Inform. Theory, vol. 51, no. 4, pp. 1486–1495, 2005.

[8] E. Martinian, G. W. Wornell, and B. Chen, ”Authentication with Distortion Criteria”, IEEE Trans. on Inform. Theory, pp. 2523–2542, July 2005.

[9] T. Kalker and F. M. Willems, ”Capacity bounds and code constructions for reversible data-hiding”, SPIE Proceedings, Security and Watermark- ing of Multimedia Contents V, vol. 5020, Santa Clara, USA, 2003.

[10] J. J. Eggers and R. Beuml and R. Tzschoppe and B. Girod, ”Inverse Mapping of SCS-Watermarked Data”, Eleventh European Signal Pro- cessing Conference EUSIPCO’2002, Toulouse, France, 2002.

[11] S. Voloshynovskiy, O. Koval, E. Topak, T. Pun, ”Message Communi- cations and Channel State Estimation for State Dependent Channels”, 27th Symposium on INFORMATION THEORY in the BENELUX, June 8-9, 2006, Noordwijk, The Netherlands.

[12] E. A. Haroutunian, ”On Bounds for E-capacity of DMC,” IEEE Trans.

on Information Theory, vol. 53, no.11, pp. 4210-4220, 2007.

[13] M. E. Haroutunian, ”Estimates of E-capacity and capacity regions for multiple-access channel with random parameter”, Lecture Notes in Computer Science, vol. 4123, Springer Verlag, 2006.

[14] E. Haroutunian, M. Haroutunian and A. Harutyunyan, ”Reliability criteria in information theory and statistical hypothesis testing”, Foun- dations and Trends in Communications and Information Theory, vol.

4, no 2-3, pp. 97-263, 2008.

[15] I. Csisz´ar and J. K¨orner, Information Theory: Coding theorems for discrete memoryless systems, Academic Press, New York, 1981.

[16] I. Csisz´ar, ”The method of types”, IEEE Trans. Inform. Theory, vol.

44, no. 6, pp. 2505–2523, 1998.

[17] P. Moulin and J. A. O’Sullivan, ”Information-theoretic analysis of information hiding”, IEEE Trans. Inform. Theory, vol. 49, no. 3, pp.

563-593, Mar. 2003.