[libav-api] Cannot open stream

Joshua Kordani jkordani at lsa2.com
Tue Jun 10 19:31:53 CEST 2014


Well I wouldn't go so far as to say you are missing something big and 
obvious.
Actually, it seems like I am.  I wasn't aware until now that the project 
provides for standing up rtsp feeds for consumption.

I was assuming that you were using a different library to start up the 
rtsp feed, consuming rtp and passing the nals to the library decoding 
calls yourself.

My questions were based on this assumption, so if you aren't doing these 
things yourself, then disregard.

Ideally, then, you shouldn't have to care about these details, but to 
close the loop...

H264 takes frames of a video and turns them into nals, packets of 
encoded data that are designed to be passed over a network.  In order to 
decode a frame from these nals, you need to collect enough of the right 
ones to decode a complete frame.  Usually this requires an sps, pps, and 
some combination of other nals that compose one frame.  Usually this is 
an idr nal alone (which is a nal that has all the data necessary to 
decode one frame) or an idr nal plus delta nals, which is to say, one 
nal that decribes a whole frame, and one or more nals who use the idr 
nal as a baseline and (presumably) carry only information that differs 
from the reference frame.

h264 servers (at the encoder, or at any machinery between it and the 
network) can be configured to provide the sps and pps nals in the middle 
of the normal bitstream, that is, you ask for data and get an sps, pps, 
and then a series of the aforementioned frame nals, with the sps and pps 
usually reintroduced periodically in order to allow new clients to begin 
decoding frames.
Sometimes though, the sps and pps nals are also/only provided in the 
rtsp setup request, and the other frame nals are only provided in the 
bitstream.  In the case of rtsp, the sps and pps nals are provided in 
the sprops-parameter-set attribute of the sdp file, you can find them in 
the wireshark paste that you provided in an earlier email.  This is 
base64 encoded binary data, which you can append immediately to an h264 
annex b (add the startcodes yourself), and then begin appending whatever 
you get from the bitstream, and you should be able to play that file in 
vlc. or the ffplay or equivalent tool provided by the libav project.

I'll save you a headache, the h264 annex b page is ridiculously... 
specified.  All it calls for is a bitstream that starts with an sps and 
pps nal, and has any other nal appended, all with 0x00000001 (so-called 
startcode) added at the beginning of each nal.  This is all you need to 
do to produce such a stream.
To consume one, you need to account for the fact that the startcode may 
be 3 or 4 null bytes followed by a 01 byte.  I forget when the startcode 
is allowed to be 3 bytes vs 4, but since I don't consume these myself, I 
just produce 4 byte versions and go about my day.  Its something like 
"all nals of type x must have a 4 byte startcode, all others may be 3 or 
4 bytes."

If you write this stream to a file and name it something.264, you should 
be able to play it in vlc.  I hear the .264 file extension is looked for 
by some players in order to figure out that it is an annex b bitstream.


Joshua Kordani
LSA Autonomy

On 6/10/14 12:17 PM, Manuel Torres wrote:
> I have to admit that I barely understood that reply so I assume that I am
> missing something big and obvious.
>
> Answers to the questions:
> "Are you aware that the sprop-parameter-sets contain the sps and pps nal
> packets encoded in base64?"
> I think that the answer would be no because I do not know what those are
> but I will look it up.
>
> "Is your code looking for the sps and pps nals in the rtp stream itself?"
> Hard to say. The library is doing everything for me. I have no clue of what
> is going on there. I set up the context, open the input, set up the codec
> and grab the frames. Nothing more advanced than that.
>
> "Are you accounting for whether or not the sps and pps are coming in-stream
> vs in the sdp?"
> I guess I am not. I do not even know how to get the data from the SDP. I
> will look it up too but I would appreciate if anybody could tell me.
>
> "If your rtp reception code can produce an h264 annex b file, can vlc play
> it?"
> I suppose you are referring to the Annex B of the H.264 specification. I
> have not read the specification but I have a copy of it so I will check
> that annex and try to create that file to test as described.
>
>
> Regardless of my answers to those questions, I understand from the reply
> that the data is there somewhere. I would be really grateful if someone
> could tell me where to find it in the library structures and, if it is not
> too much to ask, a link to how to parse "the sps and pps nal packets",
> whatever those are. Base64 is the only thing I understood from that first
> question. Shame on me.
>
>
>
> On Tue, Jun 10, 2014 at 5:28 PM, Joshua Kordani <jkordani at lsa2.com> wrote:
>
>> Are you aware that the sprop-parameter-sets contain the sps and pps nal
>> packets encoded in base64?  Is your code looking for the sps and pps nals
>> in the rtp stream itself?  Are you accounting for whether or not the sps
>> and pps are coming in-stream vs in the sdp?
>>
>> If your rtp reception code can produce an h264 annex b file, can vlc play
>> it?   Making your code create this file will tell you that you have
>> everything you need in order to pass data to the decoding library.  If vlc
>> can play the h264 annex b that you create, then you know that all you have
>> left to do is figure out how to pass this data to the library to decode it
>> yourself, most likely.
>>
>> Joshua Kordani
>> LSA Autonomy
>>
>>
>> On 6/10/14 8:11 AM, Manuel Torres wrote:
>>
>>> Captured with Wireshark:
>>>
>>> v=0
>>> o=- 1293840088672 1293840088672 IN IP4 192.168.2.93
>>> s=Live
>>> t=0 0
>>> m=audio 0 RTP/AVP 0
>>> c=IN IP4 192.168.2.93
>>> a=control:rtsp://192.168.2.93/defaultPrimary/mic0/trackID=1
>>> m=video 0 RTP/AVP 96
>>> c=IN IP4 192.168.2.93
>>> a=control:rtsp://192.168.2.93/defaultPrimary/cam0/trackID=1
>>> a=fmtp:96 packetization-mode=0; profile-level-id=42A01E;
>>> sprop-parameter-sets=J01AH42NKBaHt/4AQAA21BgYGQAAAwPoAADDUOhAB08A
>>> AQcau8uNCADp4AAg41d5cE+iwA==,KO48gA==
>>> a=rtpmap:96 H264/90000
>>> a=x-avg-params:96 source-height=1080; source-width=1920
>>> m=application 0 RTP/AVP 98
>>> c=IN IP4 192.168.2.93
>>> a=control:rtsp://192.168.2.93/defaultPrimary/metadata/trackID=1
>>> a=rtpmap:98 vnd.onvif.metadata/90000
>>>
>>>
>>>
>>> On Tue, Jun 10, 2014 at 12:58 PM, Luca Barbato <lu_zero at gentoo.org>
>>> wrote:
>>>
>>>   On 10/06/14 12:44, Manuel Torres wrote:
>>>>> Would there be any other way to get the missing data? I guess that there
>>>>> must be a way since VLC and MPlayer open the stream.
>>>>>
>>>>>   Could you please provide the sdp?
>>>> lu
>>>>
>>>> _______________________________________________
>>>> libav-api mailing list
>>>> libav-api at libav.org
>>>> https://lists.libav.org/mailman/listinfo/libav-api
>>>>
>>>>   _______________________________________________
>>> libav-api mailing list
>>> libav-api at libav.org
>>> https://lists.libav.org/mailman/listinfo/libav-api
>>>
>> _______________________________________________
>> libav-api mailing list
>> libav-api at libav.org
>> https://lists.libav.org/mailman/listinfo/libav-api
>>
> _______________________________________________
> libav-api mailing list
> libav-api at libav.org
> https://lists.libav.org/mailman/listinfo/libav-api



More information about the libav-api mailing list