66 replies to this topic

[dolphs]

Hi, I have two avm routers ( fritzbox ) with on board VPN ( IPSec ) capabilities and both are linked-up with LAN-to-LAN.

Now it appears on starting an HD stream, eg " http://192.168.20.12...:1:C00000:0:0:0 " things stutter and freeze.

On bypassing the VPN, things start to work nearly flawless ( occasional hick-up ) so that means both Internet connections are OK ( in terms of up and download speed, to be precise 100M/bit down and 30M/bit upload and 300M/bit / 100M/bit ).

 

Unfortunately on configuring its LAN-to-LAN link-up it does not leave much room for tuning / configuring.

Reading avm's service page: shows the throughput rates over VPN can be lower than the rates allowed by the respective Internet connections.

 

Now is this known and is it possible to stream approx. 20Mbit ( HD stream ) over VPN a few hours a day, or is more professional equipment needed except for just a VPN on a non-Enterprise router or raspberry/nas ? I am considering to install OpenVPN, but if I will get similar results it will be waste of time of course.

 

Meanwhile also like to find out, if these http streams ( see example above ) can be configured for https ?

 

 

Happy Easter!

Edited by dolphs, 27 March 2016 - 07:37.

[littlesat]

Arrange the vpn on your box...

[bogdanm]

Arrange the vpn on your box...

How made this ?

[Erik Slagter]

Arrange the vpn on your box...

That is not a very good idea.

 

As you're saying you're using Fritzbox, I have a hunch 

 

Although IPsec in principe is perfectly fine for streaming, you might want to try OpenVPN as well.

[dolphs]

Great, so IPSec should be OK for this matter - thanks for confirming.

In other words this needs to be addressed to avm for further investigation ( why http streams stutter ).

 

Meanwhile found out when transcoding channels back to 7,5Mbit things start to work over VPN ...

Edited by dolphs, 27 March 2016 - 09:50.

[littlesat]

I have it running here  OpenVPN (server) on my ET10K extreme smoothly with certificates as it should be... all starts with git install openvpn on your box...a lot of reading to get certificates etc... and create the right key files and configs... and of course a port forwarded from you fritz box to your box... (e.g. 443 - this works from mostly anywhere).

 

The standard fritzbox openvpn is extreme lame. This is only password protection with a not really secret keys....

[dolphs]

OK littlesat might try openvpn coming days on my et10000, might even get it to work ;-).

BTW implies forwarding rules to both TCP 443 and TCP 943, UDP 1194 imho?

[WanWizard]

The problem with streaming is not bandwidth, it's latency and jitter.

 

Another challenge can be MTU size. Streaming data are usually full-size packets. On the LAN, the MTU is typically 1500 bytes, but due to the additional VPN IP headers, the MTU side inside a VPN is smaller. Which means a single packet on the LAN becomes two packets in the tunnel, so you have fragmentation that doubles the packet rate. Which in turn may introduce latency and jitter.

 

You can test this by pinging the other side with a full package payload (the default ping packet is only 64 bytes). 

 

There is about 30% overhead on VPN connections which you need to take into account, but when you have plenty of bandwidth, that is not really an issue.

[Erik Slagter]

If you make your et10k transcode to 720, with a reasonably high bit rate, you will have quite an acceptable picture quality, but the stream is much less sensitive to latency, I discovered from real life.

[WanWizard]

Because it is no longer a transport stream, and therefore more buffering is available client side?

[Erik Slagter]

Actually it IS a transport stream  But somehow the elementary streams are more compact and the stream is (re)muxed to only include three streams, which might help.

[gorski]

I don't know if this helps [in this case] but better to mention it, than not, I guess... So, have a look and see...

https://www.smartdnsproxy.com/page/vpn-service-vs-smart-dns-proxy.aspx

https://www.smartdnsproxy.com/

My streaming is cool, esp. on ET10K...

Edited by gorski, 27 March 2016 - 14:49.

[dolphs]

@Gorski - thanks for that link, but paying for on occasional joy does not fulfill my expectations instead I could do with a 7,5Mbit stream.

I appreciate your post though! Earlier streamed BVN TV for a few hours, just for fun and that is near flawless ( occasional glitch ) but did not encounter a freeze

 

 

If you make your et10k transcode to 720, with a reasonably high bit rate, you will have quite an acceptable picture quality, but the stream is much less sensitive to latency, I discovered from real life.

 

 

Following example is working fine ( quality poort/average if you compare it to the original HD stream ):

http://192.168.20.12:8001/1:0:19:4BC9:832:600:FFFF0000:0:0:0?bitrate=7500000?aspectratio=2

It is the 7,5Mbit limit I am facing as it seems due to MTU and all sorts of issues, need to find tools / commands how to test this properly ...

Need to dive in to the parameters to set it to 720, if you have an example that would be perfect!

 

 

 

 

As you're saying you're using Fritzbox, I have a hunch 

 

 

tell me that hunch please, or did I miss something and you mean simply: avm and their stuff is kind of useless material: just tell me I can handle it  ;-)

 

 

 

The problem with streaming is not bandwidth, it's latency and jitter.

 

Another challenge can be MTU size. Streaming data are usually full-size packets. On the LAN, the MTU is typically 1500 bytes, but due to the additional VPN IP headers, the MTU side inside a VPN is smaller. Which means a single packet on the LAN becomes two packets in the tunnel, so you have fragmentation that doubles the packet rate. Which in turn may introduce latency and jitter.

 

You can test this by pinging the other side with a full package payload (the default ping packet is only 64 bytes). 

 

There is about 30% overhead on VPN connections which you need to take into account, but when you have plenty of bandwidth, that is not really an issue.

 

now and this is exactly where I need to get skilled as I have no clue how to test this properly, except doing a simple " ping " .

Both sides show around 70ms ( packet size 56 data bytes as it seems )

root@et10000:~# ping 192.168.20.12

PING 192.168.20.12 (192.168.20.12): 56 data bytes
64 bytes from 192.168.20.12: seq=0 ttl=62 time=69.685 ms
64 bytes from 192.168.20.12: seq=1 ttl=62 time=69.452 ms
64 bytes from 192.168.20.12: seq=2 ttl=62 time=69.236 ms
64 bytes from 192.168.20.12: seq=3 ttl=62 time=69.948 ms
64 bytes from 192.168.20.12: seq=4 ttl=62 time=70.196 ms
64 bytes from 192.168.20.12: seq=5 ttl=62 time=70.000 ms
64 bytes from 192.168.20.12: seq=6 ttl=62 time=69.743 ms
64 bytes from 192.168.20.12: seq=7 ttl=62 time=69.761 ms
64 bytes from 192.168.20.12: seq=8 ttl=62 time=69.988 ms
64 bytes from 192.168.20.12: seq=9 ttl=62 time=69.546 ms
64 bytes from 192.168.20.12: seq=10 ttl=62 time=70.028 ms
64 bytes from 192.168.20.12: seq=11 ttl=62 time=69.797 ms
64 bytes from 192.168.20.12: seq=12 ttl=62 time=69.834 ms
64 bytes from 192.168.20.12: seq=13 ttl=62 time=69.601 ms
64 bytes from 192.168.20.12: seq=14 ttl=62 time=69.648 ms
64 bytes from 192.168.20.12: seq=15 ttl=62 time=69.373 ms

VPN 30% less speed is indeed what I heard before, however that would mean I have still somewhat 20Mbit available ( 30% from 30Mbit ) which should be more than fine for HD streaming ( approx. 15Mbit )

[WanWizard]

The problem isn't directly bandwidth (that is something else then "speed" ) in your case, you have plenty.

 

The problem is that the box will send ethernet packets out containing 1500 bytes, the standard ethernet MTU size. Your VPN termination point (router, etc), will have to forward this packet through the VPN tunnel, but due to the header overhead of a VPN (besides the ethernet header and IP header, there's also a VPN header) the MTU size of the tunnel is less. So your router has to break the payload of the packet in two (fragment), so for every packet send my your box, two will arrive at the client side, which need to assemble all these packets again.

 

So two possible points of delay: the router that has to process each and every packet to fragment them, and the client that has to reassemble them.

 

on linux, you can use the -s parameter to set the packet size. I've tested it through an IPSec VPN: I can send packets of 1472 bytes, larger won't go through the tunnel without fragmentation.

root@et10000:~# ping 10.1.2.3 -s 1472
PING 10.1.2.3 (10.1.2.3): 1472 data bytes
1480 bytes from 10.1.2.3: seq=0 ttl=62 time=33.150 ms
1480 bytes from 10.1.2.3: seq=1 ttl=62 time=29.971 ms
1480 bytes from 10.1.2.3: seq=2 ttl=62 time=29.753 ms
1480 bytes from 10.1.2.3: seq=3 ttl=62 time=29.835 ms
^C
--- 10.1.2.3 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 29.753/30.677/33.150 ms

root@et10000:~# ping 10.1.2.3 -s 1500
PING 10.1.2.3 (10.1.2.3): 1500 data bytes
^C
--- 10.1.2.3 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet los

( this is a tunnel between a Draytek 2960 and a Draytek 2860).

 

Note that for IPsec, it also very important how you make your setup. A lot of (cheaper) home routers don't have the CPU to do proper encryption, a Fritzbox completely collapses on heavy AES, troughput doubles when you use 3DES (which is not considered secure anymore). Quite a few don't even manage 10Mbps encrypted traffic through a tunnel...

[dolphs]

thanks WanWizard, I think I reached the right person considering your nick ​.

 

Note that for IPsec, it also very important how you make your setup. A lot of (cheaper) home routers don't have the CPU to do proper encryption, a Fritzbox completely collapses on heavy AES, troughput doubles when you use 3DES (which is not considered secure anymore). Quite a few don't even manage 10Mbps encrypted traffic through a tunnel...

 

Well this is an issue I addressed towards avm already, although both routers ( 7490 and 3370 ) are dual core and do not show lots of cpu load when stream starts thru vpn, however CPU is not the main thing to watch like a dog, see later on I noticed sirq  ​goes thru the roof (100%).

 

Find below an example for a stream ( NPO1 HD which is approx 12M/bit according to the bitrate viewer tool ):

 

 

A simple top shows on both boxes while streaming ( et10000 from 7490 requests http stream from stb behind 3370 ) :

 

 

A/ omitting VPN:

 

7490:

Mem: 83452K used, 162460K free, 3812K shrd, 6004K buff, 28808K cached
CPU0:  0.7% usr  0.3% sys  0.0% nic 98.8% idle  0.0% io  0.0% irq  0.0% sirq
CPU1:  0.1% usr  0.7% sys  0.0% nic 99.0% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.00 1.00 1.00 1/126 7275
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 7275  7243 root     R     1372  0.5   1  0.4 {busybox} top
 1177     1 root     S    16060  6.5   0  0.3 /usr/bin/avm/ctlmgr
 1187     1 root     S     8340  3.3   1  0.1 upnpd

3370:

Mem: 98052K used, 16260K free, 1928K shrd, 11280K buff, 38904K cached
CPU0:  0.1% usr  0.1% sys  0.0% nic 35.6% idle  0.0% io  0.1% irq 63.7% sirq
CPU1:  0.1% usr  0.5% sys  0.0% nic 99.2% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.00 1.00 1.00 1/106 8425
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    4     2 root     SW       0  0.0   0 18.9 [ksoftirqd/0]
 8230  7847 root     R     1316  1.1   0  0.3 {busybox} top
  133     2 root     SW       0  0.0   1  0.1 [avmnet_workqueu]

nearly idle ( was expected )

 

 

B/ using a 7,5Mbit transcoded VPN connection

 

7490

Mem: 83272K used, 162640K free, 3812K shrd, 6004K buff, 28808K cached
CPU0:  0.1% usr  1.3% sys  0.0% nic 98.2% idle  0.0% io  0.0% irq  0.1% sirq
CPU1:  0.0% usr  0.7% sys  0.0% nic 99.2% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.00 1.00 1.00 1/126 7275
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 7275  7243 root     R     1372  0.5   1  0.4 {busybox} top
 1177     1 root     S    16060  6.5   0  0.2 /usr/bin/avm/ctlmgr
 1609     1 root     S     9720  3.9   1  0.0 /usr/bin/aha

3370

Mem: 98048K used, 16264K free, 1928K shrd, 11280K buff, 38904K cached
CPU0:  0.3% usr  0.5% sys  0.0% nic 97.8% idle  0.0% io  0.5% irq  0.5% sirq
CPU1:  0.3% usr  0.5% sys  0.0% nic 99.0% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.00 1.00 1.00 1/108 10788
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 8230  7847 root     R     1316  1.1   1  0.4 {busybox} top
 1033     1 root     S     3120  2.7   0  0.1 l2tpv3d
 1040     1 root     S    15136 13.2   1  0.0 /usr/bin/avm/ctlmgr

As transcoded stream is flawless this indicates the VPN can happily handle the stream.

More interesting barely high value of sirq noticed

 

 

C/ full blown VPN connection ( 12-13Mbit in this case ):

Now this is where things get interesting ...

 

 

7490

Mem: 83332K used, 162580K free, 3812K shrd, 6004K buff, 28808K cached
CPU0:  0.1% usr  0.5% sys  0.0% nic 33.6% idle  0.0% io  0.0% irq 65.5% sirq
CPU1:  0.5% usr  0.7% sys  0.0% nic 98.6% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.00 1.00 1.00 3/126 7275
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    4     2 root     RW       0  0.0   0  0.9 [ksoftirqd/0]
 1230     1 root     S     4088  1.6   0  0.6 multid
 7275  7243 root     R     1372  0.5   1  0.4 {busybox} top

3370

After a few seconds my session to this box times out, in other words it seems to collapse :-)

Mem: 99768K used, 14544K free, 1940K shrd, 11280K buff, 38916K cached
CPU0:  0.0% usr  0.0% sys  0.0% nic  0.0% idle  0.0% io  0.0% irq  100% sirq
CPU1: 14.2% usr 14.2% sys  0.0% nic 71.4% idle  0.0% io  0.0% irq  0.0% sirq
Load average: 1.51 1.24 1.09 2/112 12219
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    4     2 root     RW       0  0.0   0 60.6 [ksoftirqd/0]
12217 12180 root     R     1316  1.1   1  4.7 {busybox} top

It got to my attention the sirq time has increased massivly, on the 3370 to 100% so here we have an indication.

This sirq is time where the CPU was busy executing instructions in the context of handling soft IRQs: VPN

 

 

Now rgd ping packets both stbs return OK with 1500 packet size ( idle VPN connection ), however over double the time as the draytek connection shows ( around 70ms ):

root@et10000:~# ping 192.168.20.12 -s 1500
PING 192.168.20.12 (192.168.20.12): 1500 data bytes

1508 bytes from 192.168.20.12: seq=0 ttl=62 time=71.864 ms
1508 bytes from 192.168.20.12: seq=1 ttl=62 time=71.311 ms
1508 bytes from 192.168.20.12: seq=2 ttl=62 time=71.958 ms
1508 bytes from 192.168.20.12: seq=3 ttl=62 time=71.788 ms
1508 bytes from 192.168.20.12: seq=4 ttl=62 time=72.016 ms
1508 bytes from 192.168.20.12: seq=5 ttl=62 time=71.712 ms
1508 bytes from 192.168.20.12: seq=6 ttl=62 time=71.270 ms
1508 bytes from 192.168.20.12: seq=7 ttl=62 time=71.783 ms
1508 bytes from 192.168.20.12: seq=8 ttl=62 time=71.504 ms
1508 bytes from 192.168.20.12: seq=9 ttl=62 time=71.990 ms
1508 bytes from 192.168.20.12: seq=10 ttl=62 time=71.909 ms
1508 bytes from 192.168.20.12: seq=11 ttl=62 time=71.869 ms
1508 bytes from 192.168.20.12: seq=12 ttl=62 time=72.129 ms
1508 bytes from 192.168.20.12: seq=13 ttl=62 time=71.183 ms
1508 bytes from 192.168.20.12: seq=14 ttl=62 time=72.145 ms
1508 bytes from 192.168.20.12: seq=15 ttl=62 time=71.726 ms


root@vusolo:~# ping 192.168.10.11 -s 1500

PING 192.168.10.11 (192.168.10.11): 1500 data bytes
1508 bytes from 192.168.10.11: seq=0 ttl=62 time=74.775 ms
1508 bytes from 192.168.10.11: seq=1 ttl=62 time=72.407 ms
1508 bytes from 192.168.10.11: seq=2 ttl=62 time=71.458 ms
1508 bytes from 192.168.10.11: seq=3 ttl=62 time=71.767 ms
1508 bytes from 192.168.10.11: seq=4 ttl=62 time=72.418 ms
1508 bytes from 192.168.10.11: seq=5 ttl=62 time=72.150 ms
1508 bytes from 192.168.10.11: seq=6 ttl=62 time=71.957 ms
1508 bytes from 192.168.10.11: seq=7 ttl=62 time=71.988 ms
1508 bytes from 192.168.10.11: seq=8 ttl=62 time=72.263 ms
1508 bytes from 192.168.10.11: seq=9 ttl=62 time=71.479 ms
1508 bytes from 192.168.10.11: seq=10 ttl=62 time=71.865 ms
1508 bytes from 192.168.10.11: seq=11 ttl=62 time=71.712 ms
1508 bytes from 192.168.10.11: seq=12 ttl=62 time=71.921 ms
1508 bytes from 192.168.10.11: seq=13 ttl=62 time=72.254 ms
1508 bytes from 192.168.10.11: seq=14 ttl=62 time=72.016 ms
1508 bytes from 192.168.10.11: seq=15 ttl=62 time=71.654 ms

Things getting hard from "-s 25153 " :

root@vusolo:~# ping 192.168.10.11 -s 25152
PING 192.168.10.11 (192.168.10.11): 25152 data bytes

25160 bytes from 192.168.10.11: seq=0 ttl=62 time=100.949 ms
25160 bytes from 192.168.10.11: seq=1 ttl=62 time=100.842 ms
25160 bytes from 192.168.10.11: seq=2 ttl=62 time=100.495 ms
25160 bytes from 192.168.10.11: seq=3 ttl=62 time=100.008 ms
25160 bytes from 192.168.10.11: seq=4 ttl=62 time=100.042 ms
25160 bytes from 192.168.10.11: seq=5 ttl=62 time=99.858 ms
25160 bytes from 192.168.10.11: seq=6 ttl=62 time=99.315 ms
25160 bytes from 192.168.10.11: seq=7 ttl=62 time=99.904 ms
25160 bytes from 192.168.10.11: seq=8 ttl=62 time=99.810 ms
25160 bytes from 192.168.10.11: seq=9 ttl=62 time=99.536 ms
25160 bytes from 192.168.10.11: seq=10 ttl=62 time=100.202 ms
25160 bytes from 192.168.10.11: seq=11 ttl=62 time=99.754 ms
25160 bytes from 192.168.10.11: seq=12 ttl=62 time=99.880 ms
25160 bytes from 192.168.10.11: seq=13 ttl=62 time=100.152 ms
25160 bytes from 192.168.10.11: seq=14 ttl=62 time=99.730 ms
25160 bytes from 192.168.10.11: seq=15 ttl=62 time=100.103 ms


root@vusolo:~# ping 192.168.10.11 -s 25153
PING 192.168.10.11 (192.168.10.11): 25153 data bytes

25161 bytes from 192.168.10.11: seq=0 ttl=62 time=100.957 ms
25161 bytes from 192.168.10.11: seq=4 ttl=62 time=100.383 ms
25161 bytes from 192.168.10.11: seq=16 ttl=62 time=101.741 ms



root@et10000:~# ping 192.168.20.12 -s 25152
PING 192.168.30.12 (192.168.20.12): 25152 data bytes

25160 bytes from 192.168.20.12: seq=0 ttl=62 time=99.445 ms
25160 bytes from 192.168.20.12: seq=1 ttl=62 time=99.045 ms
25160 bytes from 192.168.20.12: seq=2 ttl=62 time=99.229 ms
25160 bytes from 192.168.20.12: seq=3 ttl=62 time=99.549 ms
25160 bytes from 192.168.20.12: seq=4 ttl=62 time=98.873 ms
25160 bytes from 192.168.20.12: seq=5 ttl=62 time=99.097 ms
25160 bytes from 192.168.20.12: seq=6 ttl=62 time=98.869 ms
25160 bytes from 192.168.20.12: seq=7 ttl=62 time=99.379 ms
25160 bytes from 192.168.20.12: seq=8 ttl=62 time=99.142 ms
25160 bytes from 192.168.20.12: seq=9 ttl=62 time=98.822 ms
25160 bytes from 192.168.20.12: seq=10 ttl=62 time=99.296 ms
25160 bytes from 192.168.20.12: seq=11 ttl=62 time=99.395 ms
25160 bytes from 192.168.20.12: seq=12 ttl=62 time=98.846 ms
25160 bytes from 192.168.20.12: seq=13 ttl=62 time=99.270 ms
25160 bytes from 192.168.20.12: seq=14 ttl=62 time=99.313 ms
25160 bytes from 192.168.20.12: seq=15 ttl=62 time=98.876 ms

Edited by dolphs, 28 March 2016 - 05:26.

[gorski]

Well, for what it's worth: the service I mentioned costs less than €45 for 2 years - and frequently one may even find a discount, so go figure - less than €2 a month (or less than €1 sometimes!)...

 

Challenge is all very well, solve it if you can, the reward is fabulous, I am sure, best of luck with that! - but at the end of the day, from my long "test" - this works!

[Erik Slagter]

Yeah, Fritzbox has a slate record of obscure bugs that don't get fixed or only after a long long time. I wouldn't recommend their devices, especially not for advanced users.

 

The syntax of the built-in transcoding interface (et10k) is like this:

 

?width
?height
?framerate
?interlaced
?aspectratio

 

Please note that every keyword needs to start with a question mark, like this: ?width=1080?height=720 not with an ampersand which is usual on http.

[dolphs]

If you make your et10k transcode to 720, with a reasonably high bit rate, you will have quite an acceptable picture quality, but the stream is much less sensitive to latency, I discovered from real life.

 

OK worth a try, let's assume this means ?width=1280?height=720

Edited by dolphs, 28 March 2016 - 11:57.

[dolphs]

Yeah, Fritzbox has a slate record of obscure bugs that don't get fixed or only after a long long time. I wouldn't recommend their devices, especially not for advanced users.

 

OK crystal clear, next time it will be Draytek if coming 6.50 OS will not solve this issue ( which I am 90% positive it won't solve this issue )

[WanWizard]

Not all Drayteks are the same either, if you're in the market for a new router, it's probably best to lookup some reviews, most have real-life VPN performance data in their tests.

 

The problem with the busybox version of ping is that you can't set the DF flag, so you can't use it to test the MTU size of the VPN. But I'm sure packet assembly will have an impact on the processing of your 3370.

 

With the Fritz, it depends very much on the model (and the power of that model), and the IPSec settings. I have one tunnel on the 2960 connected to a Fritz 7490, and I can stream an SD broadcast through the tunnel (box connected to the 7490) without any hickups (but I have 70Mbps downstream, the Fritz has 100Mbps upstream), when I try an HD channel I get artifacts. The box I used can't do transcoding, so I can't test that.