56 replies to this topic

[Stan]

With teletext subtitles, sometimes the spaces between words are missing.

 

 Sub_TTX_02.jpg   49.36KB   4 downloads

 

The issue has been mentioned by Erik some time ago in another thread.

 

[...]

Sometimes many words in a sub get concatenated without whitespace, sometimes part of the first and last character on a (long) are cropped, sometimes a line gets wrapped at an illogical place, the next word is then in dark grey ink, and the rest of the line is okay. I've seen this only on BBC. Looks like some control character is interpreted incorrectly.

 

It seems to occur when lines with different colours are displayed...

 

[littlesat]

I do not see different colors there.... But a log of enigma2 could be helpful here....

 

Here on Dutch TV we do not see this happening....

Edited by littlesat, 24 December 2014 - 13:50.

[ims]

here it works well too... (tried on recorded movies)

[Stan]

Yesterday I had it on a Freesat channel (itv2). I did record it.

 

It is strange because on the teletext page itself (888) it looks fine.

 

@littlesat

I could not find the output of the subtitles content in the enigma2 logfile. What are we looking for?

[littlesat]

It should be investigated why these spaces are removed.... wierd...

[Erik Slagter]

It's something that only shows on BBC (for me). I have changed some of the colour subtitle code which helped a lot, but it still appears now and then. I can't explain it. A slight possibility is that two lines are joined (do you have re-formatting enabled?) and no space is inserted, where it should.

[Stan]

The option "Rewrap teletext subtitles" set to "no", so I guess re-formatting is disabled.

[Erik Slagter]

Okay good to know, so we can rule that out. Do you see it often? I only notice it very rarely now.

[Stan]

I had a look at the subtitle teletext page 888 (same scene from first post).

 

 Page888_02.txt   1000bytes   19 downloads

 

Between the words there is a control character (colour definition).

 

The problem is, that this character is being removed without getting replaced by a space.

 

[littlesat]

Pare you able to give the complete delivered string here?

[ims]

I had a look at the subtitle teletext page 888 (same scene from first post).

 

Page888_02.txt

 

Between the words there is a control character (colour definition).

 

The problem is, that this character is being removed without getting replaced by a space.

provider saving chars ?

[Stan]

@littlesat

The string is in the file Page_888_02.txt posted before. Just open the file with a Hex Editor.

[ims]

0x02

Attached Files

•  _subtxt.PNG   5.87KB   3 downloads

[ims]

weird, it seems as dropped b3-b0 ... but in space in sentence only  ( 0x20 ... 0010 0000b   ... dropped b0-b3  => 0010 = 0x02 )

[Erik Slagter]

The problem is, that this character is being removed without getting replaced by a space.

Yes that is what I expected. On a teletext page the (colour) attribute characters also occupy a space (the frame buffer is very strict 40x22). So the BBC uses them as delimiter just like space would do. On the other hand, I was under the impression I handled this case correctly. But if some can find the bug please go ahead

[betacentauri]

Mhmm. Here http://sourceforge.n...letext.cpp#l397 in line 628 you check whether color has changed. It has not changed. So you do nothing. Then in line 694 the line is converted into a string. I guess in this step the "color" chars are not replaced by spaces. So I would add an else branch to line 628 and replace the color char with space.
But is it according to the spec? Or does BBC deliver wrong subtitles?

Edited by betacentauri, 25 December 2014 - 10:36.

[Erik Slagter]

I wouldn't ever accuse the BBC of being non-standard-adhering. They're usually very compliant.

 

Now I think of it, I think I "solved" the issue in "rewrapping" mode (which I use), but the issue may still be there in "non-rewrapping" mode. One of the problems is that you can't randomly replace attribute characters by spaces, because often more then one are sent in sequence. That would mean a string of spaces between two words which look ugly. The rewrap mode would drop the multi-space sequence, but non-rewrap keeps the spaces. So I guess the solution is twofold:

 

- replace every attribute character by a space

- compact multi-space sequences to one

 

Still I think it's prone to error to try to keep the original formatting at all, rendering it to a proportional font and then centering it. I think teletext should be overlayed exactly as-is (which is quite ugly but works), or all the visible text should be extracted from the page, loose all of the layout information and render it just like any other text (which is what the re-wrap code does).

Edited by Erik Slagter, 25 December 2014 - 10:46.

[Stan]

For a quick'n dirty fix I'd go with betacentauri's proposal.

 

Although it would be nice and more reliable to stick to the teletext standard and replace every contol character with space,..

In standard mode (no re-wrap) I would expect the original formatting (no matter how many spaces between words), discarding only leading and trailing spaces.

[Erik Slagter]

But then    you would  get sentences     like this, which I think is ugly. Not only need the leading and trailing whitespace to be removed, also the multiple whitespaces between words. Either that, or 100% stick to the original layout, on a 40x22 grid.

 

I will happily apply a patch that will fix this in a nice way.

[Stan]

There may be situations when multiple whitespaces are in there intentionally.

Besides, sequences of more than one control character between words are not common in subtitles..

 

One could always switch to "rewrap" mode, where multiple whitespaces are elimiinated.

 

But that's just my opinion, I'm quite confident to let you decide...

Edited by Stan, 26 December 2014 - 12:03.