15 replies to this topic

[dolphs]

Dear OpenPLi/networking/linux experts,

 

 

Meanwhile VPN ( ovpn 2.4.1 ) has been set up between two homes ( both N3150NH ).
- dhovpn01, upload 30 Mbit/ download 100Mbit- VDSL2
- buovpn01, upload 150Mbit/ download 300Mbit- fiber

 

 

But yet the tuning part kicks in, also on the stbs.

 

 

As it seems from dm8000 to vuuno4k there is just enough bandwidth to stream HD (20Mbit).
Unfortunately the other way around there is only 10Mbit available while 15 is minimum for this exercise.
- buovpn01 (dm8000) -> dhovpn01 (vuuno4k): FAIR ( 20Mbit while >80 should be possible )
- dhovpn01 (vuuno4k)-> buovpn01 (dm8000) : NOK! ( 10Mbit while >20 should be possible )

 

 

If connection is established between both ovpn servers directly it shows:
- buovpn01 -> dhovpn01 : FAIR ( 40Mbit, max download VDSL2 line 100Mbit though )
- dhovpn01 -> buovpn01 : OK   ( 27Mbit, max upload VDSL2 line )

 

 

Therefore seeking help from you guys hoping to increase transfer speed, thus all bandwidth when using 1 thread.
Also seeking advice rgd:
- Whether Hyperthreading in BIOS should be switched off ?
- configure TCP in ovpn instead of UDP considering HD-video (http) streams ?
+ Could it be somehow using TCP instead of UDP results in 10mbps troughputs ?
- tuning needed on ovpn linux kernel as well kernel on stbs ?

 

My goal is both ends should have 15-20Mbit continuously which should be sufficent for (U)HD streams to a 50" TV.
Attached iperf data ( iperf.txt ), which show results from single and 3/5 parrallel threads.

 

Hope this issue can get sorted.
PS: without VPN ( portforwarding on router ) it works flawless!

Attached Files

•  iperf.txt   6.5KB   5 downloads

[WanWizard]

The problem isn't (directly) bandwidth, the problem is latency and jitter.

 

When you stream HD, you stream the received transportstream, as and when you received it. Any hickup longer than the timing between frames, any out of sequence TCP packets or dropped UDP packets, and you've lost the stream, and the client needs to re-sync again.

 

A VPN adds quite a bit of overhead and latency. In case of oVPN, make sure you have lzo compression disabled, that sure won't help. 

[dolphs]

thanks WAN, I was hoping you would look at this thread, can I invite you to grab the iperf.txt attachment in my initial post.

Maybe it leads to a clue already what is going on, on server side your suggested parameter was included already, server shows:

 

proto udp4

sndbuf 393216
rcvbuf 393216

push "sndbuf 393216"
push "rcvbuf 393216"

 

comp-lzo no    
fast-io        

 

 

cheers!

[WanWizard]

Not really have time atm, I'm in the middle of moving house.

[dolphs]

okay no worries chief, good luck moving !

[Trial]

Hi,

have you made speed tests? VPN needs quite a bit of CPU resources. Try FTP of a 1GB files between the 2 boxes in both directions and measure the time. Do it with and without active VPN.

 

I think VPN might be the bottleneck.

 

ciao

[Erik Slagter]

Type uptime a few times (or top) while stressing OpenVPN. You will see the load will raise almost nothing.

[dolphs]

updated few kernel settings on both ends, and got some performance increase :

 

 

WAS

root@buovpn01:~# iperf -c 192.168.10.11 -p5001 -t60 -P1
------------------------------------------------------------
Client connecting to 192.168.10.11, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[  3] local 10.8.0.2 port 46666 connected with 192.168.10.11 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.2 sec   290 MBytes  40.5 Mbits/sec

 

NOW

 

 

root@buovpn01

:~# iperf -c 192.168.10.11 -p5001 -t60 -P1
------------------------------------------------------------
Client connecting to 192.168.10.11, TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  3] local 10.8.0.2 port 59982 connected with 192.168.10.11 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   515 MBytes  71.9 Mbits/sec

 

 

Between both VPNs this is acceptable for me( therotetically +10Mbit would have been perfect ), unfortunately no joy yet on the machines behind these.

That means more investigation ( kernel,etc  )is needed on the stbs ( eg tcp_rmem, tcp_wmem, wmem_max, rmem_max, etc )

Edited by dolphs, 13 May 2017 - 14:49.

[dolphs]

we are getting somewhere, now when I will be physically on the other side I can confirm it has been sorted ...

 

root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t60 -P1
 

root@dm8000:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 42769
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.1 sec   160 MBytes  22.4 Mbits/sec
[  5] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 54890
[  5]  0.0-60.5 sec   193 MBytes  26.8 Mbits/sec
[  4] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 42785
[  4]  0.0-60.0 sec   171 MBytes  23.9 Mbits/sec

 

 

 

root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t60 -P1
 

root@vuuno4k:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51269
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.2 sec   670 MBytes  93.3 Mbits/sec
[  5] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51270
[  5]  0.0-60.2 sec   654 MBytes  91.1 Mbits/sec
[  4] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51271
[  4]  0.0-60.2 sec   626 MBytes  87.3 Mbits/sec

Edited by dolphs, 13 May 2017 - 17:37.

[Erik Slagter]

updated few kernel settings on both ends, and got some performance increase :

And what would those kernel "settings" then be, that would give this "amazing" performance improvement?

[dolphs]

Except for kernel also in server conf added  " txqueuelen 1000 " - similar speed as my eth0 device.

further tweaked on ovpn servers:

 

- Increases the entropy pool from ~120 to ~4096

- Updated congestion control to tcp_htcp

- furth kernel tuning :

echo 1 > /proc/sys/net/ipv4/ip_forward

echo htcp > /proc/sys/net/ipv4/tcp_congestion_control   # default cubic
echo fq > /proc/sys/net/core/default_qdisc   # default pfifo_fast

 

## echo 0 > /proc/sys/net/ipv4/tcp_tw_reuse                     # already set
## echo 0 > /proc/sys/net/ipv4/tcp_tw_recycle                   # already set
## echo 1 > /proc/sys/net/ipv4/tcp_syncookies                   # already set

## echo 1 > /proc/sys/net/ipv4/tcp_window_scaling               # already set
## echo 1 > /proc/sys/net/ipv4/tcp_timestamps                   # already set
## echo 1 > /proc/sys/net/ipv4/tcp_sack                         # already set
echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle           # for http recommended?

echo 5000 > /proc/sys/net/core/netdev_max_backlog

 

 

Now the real real deal

 

# increase Linux autotuning TCP buffer limit to 32MB
echo 4096 87380 33554432 > /proc/sys/net/ipv4/tcp_rmem
echo 4096 87380 33554432 > /proc/sys/net/ipv4/tcp_wmem

 

# allow testing with buffers up to 64MB
echo 67108864 > /proc/sys/net/core/wmem_max
echo 67108864 > /proc/sys/net/core/rmem_max

 

 

 

Furhter both stbs updated and tweaked these kernel settings :

 

echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle   # testing

echo 5000 > /proc/sys/net/core/netdev_max_backlog

 

echo 4096 87380 16777216 > /proc/sys/net/ipv4/tcp_rmem
echo 4096 87380 16777216 > /proc/sys/net/ipv4/tcp_wmem

echo 16777216 > /proc/sys/net/core/wmem_max
echo 16777216 > /proc/sys/net/core/rmem_max

 

 

 

I'm convinced some settings are redundant but I store these for reference as it gaves me a good performance increase.

Maybe I will dive in to it further, but for now I sit back and enjoy my HD streams ...

Edited by dolphs, 14 May 2017 - 13:32.

[Erik Slagter]

I don't think they're relevant. I also think that if you repeat the tests often enough, you'll see the variation is bigger than the "improvement".

 

Messing around with tcp and buffer values is almost always futile, especially if one doesn't exactly know what it does.

Edited by Erik Slagter, 14 May 2017 - 15:06.

[dolphs]

sure and agree partly Erik, but pls explain once I revert to the defaults on both stbs the performance is horrible again ?

[Dimitrij]

echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle           # for http recommended?
echo 5000 > /proc/sys/net/core/netdev_max_backlog

# increase Linux autotuning TCP buffer limit to 32MB
echo 4096 87380 33554432 > /proc/sys/net/ipv4/tcp_rmem
echo 4096 87380 33554432 > /proc/sys/net/ipv4/tcp_wmem

# allow testing with buffers up to 64MB
echo 67108864 > /proc/sys/net/core/wmem_max
echo 67108864 > /proc/sys/net/core/rmem_max

[Erik Slagter]

The real problem is that DVB is meant to be delivered over synchronous channels. I put a bit in at one end and it comes out on the other end, after exactly x ns delay, and that goes for every bit after that. DVB streams sent over satellite, cable and terrestrial are this way exactly. Playing recordings from disk are exactly the other way around, every bit is always instantanuously available. That is also ok. But ip stream are somewhere in the middle, delays are inpredictable so you will end up with buffer overruns or underruns sooner or later, which triggers the DVB hardware to do a resync, throwing all data away until some familiar signature is found.

 

You can try to work around it, but will never work perfectly. The best achievable is to have an available bandwidth that is widely larger than the required bandwidth, and just hope the delays are within acceptable limits.

 

Or just copy the file and start playing it while it's still copying, that will work too.

[dolphs]

Hi Dimitrij, pls don't tell me these ( and more ) kernel settings will be tweaked once this package is installed?

 

enigma2-plugin-extensions-ice-network-tuner_v1.4_all.ipk 

 

Neither see it available from opli4 feeds nor oatv6, so suppose that is why you attached this package?

 

thanks