Skip to content
Advertisement

ffmpeg HLS transcoding and chunks upload to remote server

I am trying to create a so called origin server for storing the hls chunks and the manifest file on a remote server. The sending machine running ffmpeg is with IP: 192.168.178.50 and the web server which has to store both the HLS chunks and the manifest file is with IP: 192.168.178.100. My ffmpeg transcoding command line is:

$ ffmpeg -i 01_llama_drama_1080p.mp4 -profile:v high -level 5.2 -s 1920x1080 -strict -2 -start_number 0 -hls_time 6 -hls_list_size 0 -f hls http://192.168.178.100/hls/test.m3u8

ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '01_llama_drama_1080p.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2013-02-08 18:56:45
  Duration: 00:01:30.00, start: 0.000000, bitrate: 3120 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 2925 kb/s, 24 fps, 24 tbr, 48 tbn, 48 tbc (default)
    Metadata:
      creation_time   : 1970-01-01 00:00:00
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2013-02-08 18:56:46
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
[libx264 @ 0x249b880] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x249b880] profile High, level 5.2
Output #0, hls, to 'http://185.71.86.48/hls/test.m3u8':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: h264 (libx264), yuv420p, 1920x1080, q=-1--1, 24 fps, 90k tbn, 24 tbc (default)
    Metadata:
      creation_time   : 1970-01-01 00:00:00
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 libx264
    Stream #0:1(und): Audio: aac, 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2013-02-08 18:56:46
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
      encoder         : Lavc56.60.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 0x24997e0] Cannot use rename on non file protocol, this may lead to races and temporarly partial files

But no matter what method I am using I got in the access log of my apache web server the following errors:

$ tail -f /var/log/apache2/access.log

192.168.178.50 - - [27/Jun/2017:10:40:45 +0000] "POST /hls/test.m3u8 HTTP/1.1" 404 469 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:40:45 +0000] "POST /hls/test12.ts HTTP/1.1" 404 469 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:40:46 +0000] "POST /hls/test.m3u8 HTTP/1.1" 404 469 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:40:46 +0000] "POST /hls/test13.ts HTTP/1.1" 404 469 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:40:47 +0000] "POST /hls/test.m3u8 HTTP/1.1" 404 469 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:46:44 +0000] "POST /hls/test0.ts HTTP/1.1" 404 468 "-" "Lavf/56.40.101"
192.168.178.50 - - [27/Jun/2017:10:46:51 +0000] "POST /hls/test.m3u8 HTTP/1.1" 404 469 "-" "Lavf/56.40.101"

These are the supported methods on my web server:

$ nmap -Pn -p 80 --script http-methods 192.168.178.100

Starting Nmap 7.01 ( https://nmap.org ) at 2017-06-27 10:47 UTC
Nmap scan report for 192.168.178.100
Host is up (0.0015s latency).
PORT   STATE SERVICE
80/tcp open  http
| http-methods:
|_  Supported Methods: GET HEAD POST OPTIONS

Nmap done: 1 IP address (1 host up) scanned in 0.34 seconds

My web server is running apache2 on Ubuntu 16.04:

$ apache2 -v

Server version: Apache/2.4.18 (Ubuntu)
Server built:   2017-06-26T11:58:04

So according to the access.log file no matter what method I define in the ffmpeg command line it is always trying to upload the chunks using HTTP POST, but the latter is already supported and the directory I am directing the chunks has even 777 rights. Am I missing something here?

Advertisement

Answer

I have created a bash script, which is installing NGINX, configuring it and limiting the access to a certain IP network on my Github page. You can check the solution at: Origin Server running on NGINX. The point is that as @mata said HTTP POST requires some kind of a handle which to process the POST requests and save them to the storage or you can use HTTP PUT, but in this case you need to explicitly allow it, because it is by default not allowed. My solution is actually allowing HTTP PUT only in one specific directory and it is also limiting the HTTP PUT method to only one IP. Make also sure to check my WIKI page for more detailed explanation.

Advertisement