[libav-api] Problems transmitting video & data with MPEG-2 TS

Luis Díaz Más ldiaz at catec.aero
Tue Jun 28 08:49:23 CEST 2011


Hi! I'm developing a system for encoding/decoding video and metadata (in KLV
format), and I'm having some problems. At this moment I don't bother with the
KLV coding, I only want to be able to receive data in the receptor program.

The "sender" application that generates the MPEG-2 TS seems to generate
the output with video and data in separate streams. However I note weird
things:

* If I save the output to a video file when I open this file generated 
with my
"receiver" app, it reads initially all the packets in the DATA stream and
then
all the packets in the VIDEO stream. I tried to open this same video file
with
VLC, and it also seems to spend an initial time reading DATA packets and when
all of them are read then the video is shown. I'm interested in receive DATA
and VIDEO packets synchronously because in my project the data correspond to
information reported by different sensors in a UAV, and video data is
captured
with a camera.

* If I direct the output through UDP the problem is bigger. My "receiver"
app is
blocked in the function av_find_stream_info(). I tried to open the same UDP
location with ffplay and it is also blocked in this function (trowing this
error when I close the application with ctrl+c: Could not find codec
parameters). Strangely, VLC can receive video frames (I don't know what it
does
with data packets).  However, when I remove the data stream (sending only
a video
stream) the UDP protocol runs perfectly in my applications. It seems that
av_find_stream_info() can't retrieve stream information as consequence of
data
packets.

I hope some of you can help me in order to be able to send/receive Video
frames
and metadata associated to these frames using libav. I attach you part of my
code ...

...
// Generating MPEG-2 TS muxer
...
AVOutputFormat *  _of;
AVFormatContext * _tsfc;  // MPEG-2 TS Format Context
AVCodecCotext * _vCC;     // Video Codec context
AVCodecContext * _dCC;    // Data Codec Context

_of = guess_format("mpegts", NULL, NULL);
_tsfc = avformat_alloc_context();

...
// Adding Video Stream
...
_videoStream = av_new_stream(_tsfc, 0);
_vCC = _videoStream->codec;
_vCC->codec_id = CODEC_ID_MPEG2VIDEO;
_vCC->codec_type = CODEC_TYPE_VIDEO;

...
// setting parameters of video encoder
...
AVCodec *c = avcodec_find_encodec(_vCC->codec_id);
avcodec_open(_vCC, c);

...
// Adding Data Stream
...
_dataStream = av_new_stream(_tsfc, 0); // I also have probe with ID=1 (arg 2)
_dCC=_dataStream->codec;
avcodec_get_context_defaults2(_dCC, CODEC_TYPE_DATA);
_dCC->codec_type=CODEC_TYPE_DATA;
_dCC->codec_id=CODEC_ID_TEXT;

...
// Opening output
...
av_set_parameters(_tsfc, NULL);
dump_format(_tsfc, 0, filename, 1);
url_fopen(&_tsfc->pb, filename, URL_WRONLY);
av_write_header(_tsfc);

...
// Allocation of AVFrame and creation of SwSContext for converting images
...

...
// Writting packets
...
// -- Video Packet --
// Convert input image into necessary format if necessary with sws_scale
out_size = avcodec_encode_video(_vCC, _buf, _buf_size, _picout);
if (out_size > 0)
{
  AVPacket pkt;
  av_init_packet(&pkt);
  pkt.stream_index= _videoStream->index;

  if (_vCC->coded_frame->pts != AV_NOPTS_VALUE)
    pkt.pts= av_rescale_q(_vCC->coded_frame->pts, _vCC->time_base,
_videoStream->time_base);
  if (_vCC->coded_frame->key_frame)
    pkt.flags |= AV_PKT_FLAG_KEY;
  pkt.data= _buf;
  pkt.size= out_size;

  ret = av_interleaved_write_frame(_tsfc, &pkt);
}

// -- Data Packet --
AVPacket pkt;
av_init_packet(&pkt);
std::string data("test");
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index= _dataStream->index;
pkt.data = (uint8_t *) data.c_str();
pkt.size= data.length();
av_interleaved_write_frame(_tsfc, &pkt);




More information about the libav-api mailing list