Random Notes: WebRTC

WebRTC is currently supported in Chrome 21, Opera 12, Firefox 17, and IE (via ChromeFrame). Also, a native C++ version for WebRTC stack. On chrome type chrome://version in the address tab to check current version.
The key features of WebRTC is MediaStreams for accessing user's camera and mic, PeerConnection for establishing audio/video calls, and DataChannels for p2p application data transfer.

First of all, these features need to be enabled in chrome: in the address tab type chrome://flags and then Enable MediaStream and PeerConnection.

MediaStreams

A MediaStream represents a media source, containing one or more synchronized MediaStreamTracks. A MediaStream can be converted to an object URL, and passed to a <video /> or <audio /> elements to play it. Use the getUserMedia API to get a MediaStream for the webcam/mic (prompts user for consent).
In a video conference you may have multiple MediaStreams each one for one participant having Audio/Video track.

Here is an example of get MediaStream with getUserMedia that allows to capture video stream from the user camera:

<script type="text/javascript>
navigator.webkitGetUserMedia({video:true}, onGotStream, onFailedStream);

onGotStream = function(stream) {
 var url = webkitURL.createObjectURL(stream);
 video.src = url;
}
</script>
<video id="video" autoplay="autoplay"/>

Here is another example that uses <canvas /> to display a snapshot form the video stream:

<script type="text/javascript">
function onClick() {
 snapshot.getContext("2d").drawImage(video, 0, 0, canvas.width, canvas.height);
}
</script>
<video id="video" autoplay="autoplay"/>
<canvas id="canvas"></canvas>
<button onclick="onClick()">Snapshot<button>

Here is another example that combines getUserMedia and CSS to make a grayscale effect:

<style>
 grayscale {
  -webkit-filter: grayscale();
 }
</style>
<script type="text/javascript">
 function onClick() {
  video.classList.add("grayscale");
 }
</script>

PeerConnection

PeerConnection is the API for establishing audio/video calls, it provides support for P2P, Codec control, Encryption, Bandwidth Management. Before sending media between peers, a connection should first be established (which is called signaling) in order to negociate codecs, deals with proxies and other stuff. WebRTC uses JSEP (Javascript Establishment Protocol).

WebRTC signaling and media transport

In case the cloud platform hides an App Engine then the signaling is performed through JSON/XHR+Channel, in the other hand if the cloud platform hides a SIP Proxy then SIP over WebSockets is used for signaling (example: sipml5). Here is a video showing how to use Sipml5 and a SIP proxy.

The PeerConnection API is used by the clients to exchaging multiple parameters in order to setup a session:

Local Session Description that describes the configuration of the local side,
Remote Session Description that describes the configuration of the remote side,
Remote Transport Candidates describing how to connect to the remote side.

For convention, an offer refers to the initial session description sent by caller, it contains everything the caller is capable of, and an answer is the callee response to indicate the selected or negociated parameters.
Here is the code using PeerConnection API for the caller side respresing the signaling flow:

// Create a new PeerConnection and add stream
PeerConnection(config, iceCallback)
addStream(stream)

// Create local SessionDescription and apply it
createOffer(hints)
setLocalDescription(type, desc)
startIce()

// wait for response from callee

// Receive remote SessionDescription and use it
setRemoteDescription(type, desc)

Follow is the corresponding code for the callee:

// Receive call from caller

// Create PeerConnection and set description
PeerConnection(config, iceCallback)
setRemoteDescription(type, desc)

// Create local SessionDescription and apply it
createAnswer(offer, hints)
setLocalDescription(type, desc)
startIce()

bla bla