[libav-api] Problems transmitting video & data with MPEG-2 TS

Luis Díaz Más ldiaz at catec.aero
Tue Jun 28 08:49:23 CEST 2011

Hi! I'm developing a system for encoding/decoding video and metadata (in KLV
format), and I'm having some problems. At this moment I don't bother with the
KLV coding, I only want to be able to receive data in the receptor program.

The "sender" application that generates the MPEG-2 TS seems to generate
the output with video and data in separate streams. However I note weird

* If I save the output to a video file when I open this file generated 
with my
"receiver" app, it reads initially all the packets in the DATA stream and
all the packets in the VIDEO stream. I tried to open this same video file
VLC, and it also seems to spend an initial time reading DATA packets and when
all of them are read then the video is shown. I'm interested in receive DATA
and VIDEO packets synchronously because in my project the data correspond to
information reported by different sensors in a UAV, and video data is
with a camera.

* If I direct the output through UDP the problem is bigger. My "receiver"
app is
blocked in the function av_find_stream_info(). I tried to open the same UDP
location with ffplay and it is also blocked in this function (trowing this
error when I close the application with ctrl+c: Could not find codec
parameters). Strangely, VLC can receive video frames (I don't know what it
with data packets).  However, when I remove the data stream (sending only
a video
stream) the UDP protocol runs perfectly in my applications. It seems that
av_find_stream_info() can't retrieve stream information as consequence of

I hope some of you can help me in order to be able to send/receive Video
and metadata associated to these frames using libav. I attach you part of my
code ...

// Generating MPEG-2 TS muxer
AVOutputFormat *  _of;
AVFormatContext * _tsfc;  // MPEG-2 TS Format Context
AVCodecCotext * _vCC;     // Video Codec context
AVCodecContext * _dCC;    // Data Codec Context

_of = guess_format("mpegts", NULL, NULL);
_tsfc = avformat_alloc_context();

// Adding Video Stream
_videoStream = av_new_stream(_tsfc, 0);
_vCC = _videoStream->codec;
_vCC->codec_id = CODEC_ID_MPEG2VIDEO;
_vCC->codec_type = CODEC_TYPE_VIDEO;

// setting parameters of video encoder
AVCodec *c = avcodec_find_encodec(_vCC->codec_id);
avcodec_open(_vCC, c);

// Adding Data Stream
_dataStream = av_new_stream(_tsfc, 0); // I also have probe with ID=1 (arg 2)
avcodec_get_context_defaults2(_dCC, CODEC_TYPE_DATA);

// Opening output
av_set_parameters(_tsfc, NULL);
dump_format(_tsfc, 0, filename, 1);
url_fopen(&_tsfc->pb, filename, URL_WRONLY);

// Allocation of AVFrame and creation of SwSContext for converting images

// Writting packets
// -- Video Packet --
// Convert input image into necessary format if necessary with sws_scale
out_size = avcodec_encode_video(_vCC, _buf, _buf_size, _picout);
if (out_size > 0)
  AVPacket pkt;
  pkt.stream_index= _videoStream->index;

  if (_vCC->coded_frame->pts != AV_NOPTS_VALUE)
    pkt.pts= av_rescale_q(_vCC->coded_frame->pts, _vCC->time_base,
  if (_vCC->coded_frame->key_frame)
    pkt.flags |= AV_PKT_FLAG_KEY;
  pkt.data= _buf;
  pkt.size= out_size;

  ret = av_interleaved_write_frame(_tsfc, &pkt);

// -- Data Packet --
AVPacket pkt;
std::string data("test");
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index= _dataStream->index;
pkt.data = (uint8_t *) data.c_str();
pkt.size= data.length();
av_interleaved_write_frame(_tsfc, &pkt);

More information about the libav-api mailing list