Skip to content

Switch Camera Source Mode

This demo changes the camera capture mode while the pipeline is running. The mode is controlled by the caps after v4l2src:

1
video/x-raw,format=YUY2,width=640,height=480,framerate=30/1

In the Python code, this caps is placed on a named capsfilter:

1
2
3
"v4l2src device=/dev/video0 ! "
"capsfilter name=source_filter "
"caps=video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! "

When we want to switch camera mode, we change the capsfilter to a new supported camera mode:

1
2
3
4
5
new_caps = Gst.Caps.from_string(
    f"video/x-raw,format=YUY2,width={width},height={height},framerate=30/1"
)

source_filter.set_property("caps", new_caps)

For example, the demo switches between:

1
2
640x480 @ 30 FPS
640x360 @ 30 FPS

Why Pause And Play

A camera source cannot always change resolution while it is actively pushing frames. The camera driver and GStreamer need a moment to renegotiate the stream format.

So the demo does this:

1
2
3
wait_state(Gst.State.PAUSED)
source_filter.set_property("caps", new_caps)
wait_state(Gst.State.PLAYING)

The important idea:

1
2
3
4
5
6
7
PLAYING
  ↓
PAUSED      stop active streaming, keep pipeline ready
  ↓
change caps
  ↓
PLAYING     start streaming again with the new camera mode

PAUSED is useful because the pipeline is not destroyed. The elements stay created, but streaming is stopped enough for the source caps to change.

Encoder Bitrate

When the resolution changes, the required bitrate usually changes too. A smaller frame can use a lower bitrate:

1
encoder.set_property("bitrate", bitrate_kbps)

In the demo:

1
2
switch_camera_mode(640, 360, 200)
switch_camera_mode(640, 480, 300)

Force Keyframe

After changing the camera mode, the receiver may need a fresh full H.264 frame. The demo sends a force keyframe event to the encoder:

1
force_keyframe()

This helps the receiver recover quickly after the mode change instead of waiting for the next normal keyframe.

Know Which Camera Modes Are Supported

Before changing caps, check which modes the camera supports.

Use v4l2-ctl:

1
v4l2-ctl -d /dev/video0 --list-formats-ext

Example output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
[0]: 'MJPG' (Motion-JPEG, compressed)
    Size: Discrete 1280x720
        Interval: Discrete 0.033s (30.000 fps)
    Size: Discrete 640x480
        Interval: Discrete 0.033s (30.000 fps)

[1]: 'YUYV' (YUYV 4:2:2)
    Size: Discrete 640x480
        Interval: Discrete 0.033s (30.000 fps)
    Size: Discrete 640x360
        Interval: Discrete 0.033s (30.000 fps)

The GStreamer caps must match one of these supported modes.

For YUYV, GStreamer usually uses the raw format name YUY2:

1
video/x-raw,format=YUY2,width=640,height=480,framerate=30/1

If the camera does not support the requested format, resolution, or FPS, the pipeline may fail to renegotiate or return an error.

Demo Code

demo.py 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
import gi
import time

gi.require_version("Gst", "1.0")
gi.require_version("GstVideo", "1.0")

from gi.repository import Gst, GstVideo

Gst.init(None)

pipeline_str = (
    "v4l2src device=/dev/video0 ! "
    "capsfilter name=source_filter "
    "caps=video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! "
    "videoconvert ! "
    "x264enc name=encoder "
    "bitrate=300 speed-preset=ultrafast tune=zerolatency "
    "key-int-max=30 bframes=0 byte-stream=true ! "
    "h264parse config-interval=1 ! "
    "rtph264pay pt=96 mtu=1400 config-interval=1 ! "
    "udpsink host=127.0.0.1 port=5600 sync=false async=false"
)

pipeline = Gst.parse_launch(pipeline_str)

source_filter = pipeline.get_by_name("source_filter")
encoder = pipeline.get_by_name("encoder")


def force_keyframe():
    sinkpad = encoder.get_static_pad("sink")

    event = GstVideo.video_event_new_downstream_force_key_unit(
        Gst.CLOCK_TIME_NONE,
        Gst.CLOCK_TIME_NONE,
        0,
        True,
        0,
    )

    sinkpad.send_event(event)


def wait_state(state):
    pipeline.set_state(state)
    ret, current, pending = pipeline.get_state(5 * Gst.SECOND)

    if ret == Gst.StateChangeReturn.FAILURE:
        raise RuntimeError(f"Failed to change pipeline state to {state.value_nick}")


def switch_camera_mode(width, height, bitrate_kbps):
    print(f"\n[SWITCH] camera={width}x{height}, bitrate={bitrate_kbps} kbps")

    # Pause the pipeline so v4l2src can renegotiate camera caps
    wait_state(Gst.State.PAUSED)

    new_caps = Gst.Caps.from_string(
        f"video/x-raw,format=YUY2,width={width},height={height},framerate=30/1"
    )

    source_filter.set_property("caps", new_caps)
    encoder.set_property("bitrate", bitrate_kbps)

    # Resume streaming
    wait_state(Gst.State.PLAYING)

    # Help receiver update after mode change
    force_keyframe()


try:
    wait_state(Gst.State.PLAYING)
    print("Sender started: 640x480 @ 300 kbps")

    while True:
        time.sleep(6)
        switch_camera_mode(640, 360, 200)

        time.sleep(6)
        switch_camera_mode(640, 480, 300)

except KeyboardInterrupt:
    print("\nStopping...")

finally:
    pipeline.set_state(Gst.State.NULL)

Receiver Pipeline

The demo sends RTP/H.264 over UDP to port 5600. Run this receiver in another terminal before starting the Python sender:

1
2
3
4
5
6
7
gst-launch-1.0 -v \
  udpsrc port=5600 caps="application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000" ! \
  rtph264depay ! \
  h264parse ! \
  avdec_h264 ! \
  videoconvert ! \
  autovideosink sync=false

If the sender uses host=127.0.0.1, the stream stays on the loopback interface. To receive the stream on another machine, change the sender udpsink host to the receiver machine IP address and keep the same port.

Measure Network Usage With iftop

Use iftop to watch the UDP traffic while the demo switches between camera modes and encoder bitrates.

For the default local sender, monitor the loopback interface:

1
sudo iftop -i lo -f "udp port 5600"

For a sender that streams to another machine, replace the interface with the network interface used for that route, for example:

1
sudo iftop -i eth0 -f "udp port 5600"

While the demo is running, the measured bandwidth should drop when the pipeline switches to 640x360 with 200 kbps and rise again when it switches back to 640x480 with 300 kbps. The values shown by iftop include packet overhead, so they will not exactly match the encoder bitrate.