Streaming POC with Node, Rails, RTSP and FFMPEG

Streaming is the future. Everyone is doing it. I wanted to get a streaming service Proof of Concept online to see how easy it would be to implement. Below details the services I used as well as how the project was implemented. Note the latency of the current project as it stands is very high. Going socket to socket would be quicker, but I had some issues implementing a “streamer” and “viewer” implementation for single user broadcasting.

Services

RTSP Server: RTSP Server (for incoming and outgoing streams, note the connection is via RTMP)
Rails: Rails Web Server (Views)
Node Socket Server: Nodejs Socket Server (for transcoding incoming webcam streams into rtsp input)
FFMPEG Service: FFMPEG Service after the incoming stream is started to transcode from the rtsp server to the viewer
Node Viewer Server: Nodejs for client streaming of m3u8 data (HLS server to serve data)
(DB: MariaDB for database)

Frontend Libraries

Videojs
Socket.io (Client & Server)

How it works.

In the most basic explanation here is how the service runs step by step:

Rails Server serves a webpage at / with javascript that alows the streamer (client) to open a socket connection to Node Socket Server to initiate the stream.
Once the connection has been initiated from the streamer’s webpage (who has allowed their webcam/mic to be used in the webpage) to the Node Socket Server, all data sent from the streamer to the Node Socket Server gets transcoded with FFMPEG and sent to the RTSP Server via RTMP (note the transcoding is done server side)
At this point the data is streaming from the streamers webcam in to the RTSP Server, the streamer is “live” at this point as in their data is being sent to the RTSP Server.
After the data has started streaming in we have to start our FFMPEG Service to connect to the RTSP Server to transcode the stream to m3u8 for a viewer readable format.
The viewer connects to the Rails Server at /stream which is just a basic videojs player for viewing of the stream.
The /stream page videojs connects to the Node Viewer Server where the data (.ts files and the m3u8 format) served is the output from our FFMPEG Service

Commands

# Prereq (rails project)
# Spins up Database
docker-compose up -d db
rake db:migrate

# Terminal 1 (rails project)
## Spins up rtsp server
docker-compose up rtsp

# Terminal 2 (rails project)
## Assets, lots of assets
./bin/webpack-dev-server

# Terminal 3 (rails project)
## Starts Rails server
rails s

# Terminal 4 (nodejs socket server)
## FFMPEG1's Socket input from streamer to RTSP
nodejs server.js

# Terminal 5 (nodejs viewer server)
## Starts the server for clients to conenct videojs to
nodejs main.js

# Terminal 6 (after the streamer has started their stream)
# Converts the Stream to videojs friendly HLS format
ffmpeg -v verbose -i rtmp://localhost:1935/live/test0 -c:v libx264 -c:a aac -ac 1 -strict -2 -crf 18 -profile:v baseline -maxrate 400k -bufsize 1835k -pix_fmt yuv420p -flags -global_header -hls_time 10 -hls_list_size 6 -hls_wrap 10 -start_number 1 $(pwd)/test0.m3u8

Current Situation

The Node* web services need to be consolidated.
CORS is currently wide open
The FFMPEG Service that transcodes RTMP to m3u8 needs to start once the streamer goes live. Right now this is a manual process (note this project was proof of concept)
Authentication for connecting to streams, and authorization for connecting to streams
Viewer doesn’t work in Chrome
Moving away from inline JS to React (again, POC)

Gotchas

Server socket connection needs to be the Socket.IO client version

A lesson in browser updates and videojs hls:

Native players and video.js HLS do not play nice together. I thought my video.js was out of date so I ran yarn upgrade video.js and even npm update video.js. Neither of these fixed my issue. It turned out to be an issue with my browser and a configuration setting for the hls player. The issue was reported here with a solution.

This can be solved with the following code:

    const videoJsOptions = {
        autoplay: true,
        controls: true,
        fluid: true,
        aspectRatio: '16:9',
        // THIS HLS OPTION FOR THE PLAYER TO OVERRIDE THE NATIVE PLAYER IS KEY!
        hls: {
          overrideNative: true
        },
        nativeAudioTracks: false,
        nativeVideoTracks: false,
        liveui: true,
        preload: 'auto',
        sources: [{
          src: 'http://localhost:8000/location.m3u8',
          type: 'application/x-mpegURL'
        }]
    }

After this option was added, I was still running into issues on Chrome. I continued to see a videojs error 3 media format corrupted. The player was working in Firefox, but not in Vivaldi (Chrome).

After a quick search, I realized my browser was out of date (76 and needed to go to 78). This was easily solved by updating my browser. After the browser update, the player was working as expected and the live stream was coming through properly.

Links

Without these I would have made it basically nowhere:

Jack Moore

Email: jack(at)jmoore53.com