Hello there, and a warm welcome to the Membrane tutorials. We're glad you chose to learn Membrane; and we'd like to invite you on a journey around multimedia with us, where we explore how to utilize the Membrane Framework to build applications that process audio, video, and other multimedia content in interesting ways.
What is Membrane?
Membrane is a multimedia processing framework that focuses on reliability, concurrency, and scalability. It is primarily written in Elixir, while some platform-specific or time-constrained parts are written in Rust C. With a range of existing packages and an easy-to-use interface for writing your own, Membrane can be used to process almost any type of multimedia, for example:
- stream via WebRTC, RTSP, RTMP, HLS, HTTP and other protocols,
- transcode, mix and apply custom processing of video & audio,
- accept and generate / record to MP4, MKV, FLV and other containers,
- handle dynamically connecting and disconnecting streams,
- seamlessly scale and recover from errors,
- do whatever you imagine if you implement it yourself :D Membrane makes it easy to plug in your code at almost any point of processing.
If the abbreviations above don't ring any bells, don't worry - this tutorial doesn't require any multimedia-specific knowledge. Then, other tutorials and demos will introduce you to the multimedia world!
Structure of the framework
The most basic media processing entities of Membrane are
Elements. An element might be able, for example, to mux incoming audio and video streams into MP4, or play raw audio using your sound card. You can create elements yourself, or choose from the ones provided by the framework.
Elements can be organized into a pipeline - a sequence of linked elements that perform a specific task. For example, a pipeline might receive an incoming RTSP stream from a webcam and convert it to an HLS stream, or act as a selective forwarding unit (SFU) to implement your own videoconferencing room. You'll see how to create a pipeline in the subsequent chapter.
To embrace modularity, Membrane is delivered to you in multiple packages, including plugins, formats, core and standalone libraries. The list of all Membrane packages is available here. It contains all the packages maintained by the Membrane team and some third-party packages.
Plugins provide elements that you can use in your pipeline. Each plugin lives in a
membrane_X_plugin repository, where
X can be a protocol, codec, container or functionality, for example mebrane_opus_plugin. Plugins wrapping a tool or library are named
membrane_X_LIBRARYNAME_plugin or just
membrane_LIBRARYNAME_plugin, like membrane_mp3_mad_plugin. Plugins are published on hex.pm, for example hex.pm/packages/membrane_opus_plugin and docs are at hexdocs, like hexdocs.pm/membrane_opus_plugin. Some plugins require native libraries installed in your OS. Those requirements, along with usage examples are outlined in each plugin's readme.
Apart from plugins, Membrane has stream formats, which live in
membrane_X_format repositories, where
X is usually a codec or container, for example, mebrane_opus_format. Stream formats are published the same way as packages and are used by elements to define what kind of stream can be sent or received. They also provide utility functions to deal with a given codec/container.
Membrane Core is the heart and soul of the Membrane Framework. It is written entirely in Elixir and provides the internal mechanisms and API that allow you to prepare processing elements and link them together in a convenient yet reliable way. Note that Membrane Core does not contain any multimedia-specific logic. The documentation for the developer's API is available at hexdocs.
Where can I learn Membrane?
There are a number of resources available for learning about Membrane:
The following sections in that guide will introduce the main concepts of creating Membrane elements and pipelines, without focusing on the specific details of multimedia processing.
The membrane_demo repository contains many projects, scripts and livebooks that cover different use cases of the framework. It's a good place to learn by example.
For a step-by-step guide to implementing a specific system using Membrane, check out our tutorials.
For more detailed information, you can refer to the Membrane Core documentation and the documentation for the Membrane packages maintained by the Membrane team, both of which can be accessed here.
If you see something requiring improvement in this guide, feel free to create an issue or open a PR in the membrane_tutorials repository.