• Aucun résultat trouvé

Eliminating the Back-Tracking Step in the Longest Common Subsequence (LCSS) Algorithm for Video Sequence Matching


Academic year: 2022

Partager "Eliminating the Back-Tracking Step in the Longest Common Subsequence (LCSS) Algorithm for Video Sequence Matching"

En savoir plus ( Page)

Texte intégral



Werner Bailer


Institute of Information Systems & Information Management Steyrergasse 17, 8010 Graz, Austria



Specific distance measures have been proposed in order to identify video sequences that are very similar over time but not identical (e.g. repeated takes). One such measure is based on the Longest Common Subsequence (LCSS) algorithm, a variant of the string edit distance. After building a match- ing matrix back-tracking is performed to identify the longest common subsequence. The modification for video sequence matching is that all sufficiently long subsequences that have gaps below a certain threshold need to be found. This in- creases the effort for back-tracking from linear to quadratic in the average case. In this paper we propose to eliminate the back-tracking step and integrate finding of the matching subsequences into the matrix creation with only linear effort.


A collection of video material often has a high degree of re- dundancy, not only due to the reuse of identical video seg- ments, but also because there are segments that show nearly identical content. Examples are several takes of a scene in rushes material or an event recorded from several very simi- larly positioned cameras, which is a typical case in news cov- ered by different broadcasters. These video segments do not only share similar static properties (e.g. color distribution in a frame), but are also similar over time (e.g. camera motion, movement of actors and objects). But they are not identi- cal: objects are at slightly different positions and the tem- poral alignment of the segment may vary, i.e. there may be omissions and insertions.

A measure based on the Longest Common Subsequence (LCSS) algorithm for the distance between two feature se- quencesA andB of videos has been proposed in [1]. The LCSS algorithm is a variant of the string edit distance, sup- porting gaps in the match. The authors of [2] apply the LCSS algorithm to matching trajectories in 2D space and introduce the following thresholds: a real number that defines the matching threshold between the non-discrete elements of the sequences and an integerδthat defines the maximum offset in

the positions to be matched. In order to adapt the LCSS algo- rithm to matching video segments the following modifications are made. As each element of the sequence is a multidimen- sional feature vector, a vectorθsim= (1, . . . , K)is defined, that contains the matching thresholds for allKfeatures. Sim- ilarly a vectorW = (w1, . . . , wK)representing the relative weights (P

kwk = 1) of the features is introduced. The off- setδintroduced in [2] is not relevant for this problem, as the matching subsequences can be anywhere in the parts. Instead the maximum gap sizeγof the subsequence is introduced as constraint.


In many dynamic programming algorithms, including the orig- inal LCSS algorithm (cf. [3]), am×nmatrix (wheremand ndenote the lengths of sequencesAandB respectively) is built from matching the elements of the input sequencesA andB in a first step, taking time O(nm). An example of such a matrix is shown in Figure 1. In the original LCSS al- gorithm back-tracking of the longest sequence is performed, starting at the lower right corner of the matrix and following a possible sequence to the upper left corner in timeO(m+n).

In our algorithm such a sequence might have gaps> γ thus we need to find alternative sequences that might have smaller gaps. Thus we (i) have to follow all equally long paths (i.e.

if at any cell in the matrix going up or left yields and equally long result we have to try both) and (ii) we have to try all suf- ficiently long sequences (i.e. follow all sequences that end in a value≥θlenin the bottom row or right column).

The consequence of (i) is that a back-tracking step takes in the worst caseO((m+n) log(m+n)), as we have to fol- low two possible paths at every cell in the matrix. This worst case happens when the distance between two similar station- ary sequences is determined and each element ofAmatches each element of B. The consequence of (ii) is that back- tracking has to be done for m+nsequences. In practical cases of course not all sequences have a length≥θlen. How- ever, asθlen m ≈ n for longer sequences, the number


1 1 1

2 3 3

4 4

Fig. 1. Simplified example of the matching matrix (“ctable”) of LCSS matching (adopted from [3]). The values of the one dimensional feature sequences (shown at the top and on the left) are matched with thresholdsθsim = 0.5andθlen = 3.

The LCSS algorithm yields the sequence shown with empha- sized border as the longest match. However, if we require a maximum gapγ ≤ 1, this sequence is not a valid result, as there is a gap of two between the first two matching elements.

Instead the sequences shown in gray are considered, which are results of the same length that additionally satisfy the gap constraint.

of sequences for which back-tracking has to be performed is only reduced by a small constant factor (typically around 5).

Thus the total effort for the back-tracking step in our algo- rithm is in the average caseO((m+n)2)and in the worst caseO((m+n)2log(m+n)).

In addition we might get partly overlapping matches that need to be post-processed, adding an additional effort ofO(k2) for the (usually small) numberkof matches.


For our problem we are not interested in the exact alignment of the two sequences, but only in matching subsequences that have gaps≤γ. We can thus eliminate the costly back-tracking step during the construction of the matching matrix as fol- lows. This matrix is built starting in the upper left corner, ei- ther line-by-line or column-by-column (depending on whether we match AtoB or B to A). We assume without loss of generality (the distance is symmetric) that we are working column-by-column. We keep a list of matching subsequences found so far. For every lineiwe store the column index of the last matchjlast found on this line and the matching sub- sequence this match belongs to.

Every time we encounter a matching element, we check whether it continues one of the matching subsequences. If this is the case, the city block distance L1(·) between the

new match at (i, j) and any of the previous matches must be≤ (γ + 2)1. Thus we find the closest match in the list of matches per line. We only have to check the distances L1((i, j),(ilast, jlast))for∀ilast ∈[i−(γ+ 1), i], i.e. this check can be done in constant time (withγ being typically very small). The new matching element is added to the clos- est matching subsequence and the list entries are updated.

These checks and updates are done for every matching ele- ment. In the average case we can assume that each element of Amatches none or one fromB, yieldingmin(m, n)matches and an effort for the checks ofO(min(m, n)). In the worst case described above, where each element ofAmatches every element ofBthe effort isO(mn).

As the identified matching subsequences could be too short (length≤θlen) we need to post-process the list of matching subsequences and remove the short ones, requiring an effort ofO(k).


When applying the LCSS algorithm to matching video se- quences the maximum length of gaps needs to be constrained.

The solution of the algorithm is thus not the single longest common subsequence but all sufficiently long subsequence with gaps shorter then a threshold. This modification increases the effort for the back-tracking step.

In this paper we propose to eliminate the back-tracking step but find the matching subsequences while creating the matching matrix. This requiresO(min(m, n))in the average case instead of O((m+n)2)for the back-tracking. In the worst case the effort increases toO(mn), but still less than O((m+n)2log(m+n))for back-tracking in the worst case.

In addition the effort for post-processing thekmatching sub- sequences is reduced fromO(k2)toO(k).


[1] Werner Bailer, Felix Lee, and Georg Thallinger, “Detect- ing and clustering multiple takes of one scene,” in Pro- ceedings of 14th Multimedia Modeling Conference, Ky- oto, JP, Jan. 2008, pp. 80–89.

[2] Dimitrios Gunopoulos Michail Vlachos, George Kollios,

“Discovering similar multidimensional trajectories,” in ICDE ’02: Proceedings of the 18th International Confer- ence on Data Engineering, San Jose, CA, USA, 2002, pp.

673–684, IEEE Computer Society.

[3] Thomas H. Cormen, Charles E. Leiserson, Ronald L.

Rivest, and Clifford Stein, Introduction to Algorithms, The MIT Press, 2nd edition, 2001.

1The value 2 comes from the fact that subsequent matches are connected diagonally, which corresponds to aL1distance of 2.


Documents relatifs

For each node p, we create an auxiliary node size that stores the values num(p) and points to the strings it represents in s.. When for the vertex p we have num(p) = b, this means

Observe that a (plain) sequence without any arc can be viewed as an arc-annotated sequence with an empty arc set... Different pattern matching and motif search problems have

Abstract. Our paper presents a new method for solving the rectangle piece jig- saw puzzle problem. The puzzle image is RGB full color and because of uni- form shape of the

We discard the top 5 visual words, vi- sual words containing multiple feature vectors from a single image and visual words with less than 5 feature vectors.. We use Bundler

La question n'est plus de savoir si la profession agricole va accepter d'intégrer les préoccupations environnementales mais comment elle va pouvoir se construire

Pierre Clément a alors rappelé que dans les amphis où il effectue ce type de travail pratique (en introduction à des cours sur la vision pour des psychologues, ou dans le cadre

The second one contains a basic matching algorithm only based on the visual descriptors and the points are updated according to the global object motion.. In both cases,

39 أنىيخجو نحب ةدان حوارتً ( ذباز ٌىؾ اهل ثاهاُب ًم تُهورتىلالؤ تمطبلا 128 و 160 تلاظشلا ًم زخإج )ذًاب يأ نئ ىتخ ،تكذب اهيلن فشهخلاو

Que telle a it été la conception de certains évêques des IXe-Xe siècles, c'est ce que m ani­ festent les translations collectives dans la cathé­ drale des corps

As for the content of the paper, the next section (Section 2) establishes a pathwise representation for the length of the longest common and increasing subsequence of the two words as

Le contexte éducatif, notamment scolaire cette fois, devient plus dur, tant pour les élèves (accroissement des inégalités, fragilisation, perte de sens, etc.) que pour les

Alors que le premier récolement décennal des musées avait atteint son terme au 31 décembre 2015, le rapport rendait compte, pour la première fois, d’un état détaillé et

In particular, the Repetition Free Longest Common Subsequence problem (RFLCS) is a variant of the LCS problem that asks for a longest common subsequence of two input strings with

Jusqu’à maintenant les questions et les critères d’inputs étaient définis par les membres de la direction de la Fondation. Si je prends les critères présentés

wieder auf das quantitative Kriterium gegriffen, denn laut Bundesgericht liegt es nahe – und das selbst dann, wenn die ausländische Strafver- folgungsbehörde die Vortat nicht

Starting from general mixture theory we proposed an extended shallow fluid–solid flow model, that consists of evolution equations for depth-averaged mass, depth- averaged

Our Contributions. In this paper, we study the problem of pri- vately computing the LIS in a time-bounded stream of length T. 1) Our proposed solutions compute the length of the

We give a one-pass streaming algorithm that uses O(k log m) space to compute the length of the longest increasing subsequence for a given input stream, where m ≥ max x i is an

En opposition à la théorie des mutations soma- tiques, la théorie du champ d’organisation tis- sulaire défend les notions selon lesquelles (1) le cancer est une maladie des

We thus use a similar approach for sub- sequence matching: given a window size w, we preprocess the pattern, in order to obtain a minimal 3nite-state automaton which then runs in time

Figure 5 shows that (i) the preprocessing of our rst method is quite dependent of the pattern, but after the preprocessing our algorithm runs in constant time equal to the length of

3) Source number: the source number is one of the oldest questions in signal analysis and remains of huge interest for the application of source separation methods to real

As we may notice, many components were introduced, the best-next, probabilistic and expectation guides, multiple search strategies (greedy, beam-search, anytime column search,