封装成MP4格式

1.需要做一下修改，主要是在MPEG4Writer中报错csd找不到，解决方案参考如下：

http://stackoverflow.com/questions/27302280/android-how-to-use-mediamuxer-with-video-mp4v-es-instead-of-video-avc

http://code.google.com/p/android/issues/detail?id=90138

https://android-review.googlesource.com/#/c/120945/1/media/libstagefright/Utils.cpp

https://github.com/CyanogenMod/android_frameworks_av/commit/0f730ffb77bce9cdaa95dc04625ced14fe193757

Android: How to use MediaMuxer with video/mp4v-es instead of video/avc?

Ask Question

up vote 3 down vote favorite 2

I want to be able to use mp4v-es instead of avc on some devices. The encoder runs fine using avc, but when I replace it with mp4v-es, the muxer reports: E/MPEG4Writer(12517): Missing codec specific data as in MediaMuxer error "Failed to stop the muxer", and the video cannot be played. The difference is that I am adding the correct track/format to the muxer, without receiving any error: ...else if (encoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { MediaFormat newFormat = encoder.getOutputFormat(); mTrackIndex[encID] = mMuxer.addTrack(newFormat); Is there any difference in handling mp4v-es compared to avc? One mention, I just skip "bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG" when it occurs, as for avc it was not needed.Thanks. android  video-encoding  mediacodec  mediamuxer share improve this question edited Dec 27 '14 at 19:03 mstorsjo 7,176 2 13 32 asked  Dec 4 '14 at 19:23 user1592546 628 1 7 24

Does newFormat have csd-0 and csd-1? –  Marlon  Dec 5 '14 at 7:00 1   @Marlon: the newFormat is: {height=720, mime=video/mp4v-es, csd-0=java.nio.ByteArrayBuffer[position=0,limit=30,capacity=‌​30], what=1869968451, width=1280} and csd-0: 000001B006000001B58913000001000000012000C48881F4528045A1463F‌​. csd-1 is not present, but I think it appears only for H264. –  user1592546  Dec 5 '14 at 7:39  stackoverflow.com/questions/21341169/… it looks like csd-1 could be required –  Marlon  Dec 5 '14 at 7:42 1   From an encoder to MPEG4Writer, I don't think it is mandatory to have 2 buffers for csd. MPEG4Writer can handle only one buffer. The error i.e. Missing codec specific data comes when there is no CSD. For video/mp4v-es i.e. MPEG4 video elementary stream, the MPEG4Writer expects the data to be packaged as ESDS format as compared to AVCC as here: androidxref.com/5.0.0_r2/xref/frameworks/av/media/… –  Ganesh  Dec 15 '14 at 0:25 1   The csd is read when a new Track is created (Ref: MPEG4Writer.cpp above, line no.1370). For the track, video encoder is the source and hence, your encoder should support getFormat in which the data should be packaged in ESDS format. –  Ganesh  Dec 15 '14 at 0:33 show 2 more comments

2 Answers

active oldest votes

up vote 2 down vote accepted

Just as Ganesh pointed out, unfortunately it does seem that this isn't possible right now, without modifying the platform source. There's actually two ways that the codec specific data can be passed to the internal MPEG4Writer class, but neither of them actually work without modifications. As Ganesh found, the logic for remapping MediaFormat keys to the internal format seems to be missing handling of codec specific data for any other video codec than H264. A tested modification that fixes this issue is as follows: diff --git a/media/libstagefright/Utils.cpp b/media/libstagefright/Utils.cpp index 25afc5b..304fe59 100644 --- a/media/libstagefright/Utils.cpp +++ b/media/libstagefright/Utils.cpp @@ -549,14 +549,14 @@ void convertMessageToMetaData(const sp<AMessage> &msg, sp<MetaData> &meta) { // reassemble the csd data into its original form sp<ABuffer> csd0; if (msg->findBuffer("csd-0", &csd0)) { - if (mime.startsWith("video/")) { // do we need to be stricter than this? + if (mime == MEDIA_MIMETYPE_VIDEO_AVC) { sp<ABuffer> csd1; if (msg->findBuffer("csd-1", &csd1)) { char avcc[1024]; // that oughta be enough, right? size_t outsize = reassembleAVCC(csd0, csd1, avcc); meta->setData(kKeyAVCC, kKeyAVCC, avcc, outsize); } - } else if (mime.startsWith("audio/")) { + } else if (mime == MEDIA_MIMETYPE_AUDIO_AAC || mime == MEDIA_MIMETYPE_VIDEO_MPEG4) { int csd0size = csd0->size(); char esds[csd0size + 31]; reassembleESDS(csd0, esds); Secondly, instead of passing the codec specific data as csd-0 in MediaFormat, you could in principle pass the same buffer (with the MediaCodec.BUFFER_FLAG_CODEC_CONFIG flag set) to MediaMuxer.writeSampleData. This approach doesn't work currently since this method doesn't check for the codec config flag at all - it could be fixed with this modification: diff --git a/media/libstagefright/MediaMuxer.cpp b/media/libstagefright/MediaMuxer.cpp index c7c6f34..d612e01 100644 --- a/media/libstagefright/MediaMuxer.cpp +++ b/media/libstagefright/MediaMuxer.cpp @@ -193,6 +193,9 @@ status_t MediaMuxer::writeSampleData(const sp<ABuffer> &buffer, size_t trackInde if (flags & MediaCodec::BUFFER_FLAG_SYNCFRAME) { sampleMetaData->setInt32(kKeyIsSyncFrame, true); } + if (flags & MediaCodec::BUFFER_FLAG_CODECCONFIG) { + sampleMetaData->setInt32(kKeyIsCodecConfig, true); + } sp<MediaAdapter> currentTrack = mTrackList[trackIndex]; // This pushBuffer will wait until the mediaBuffer is consumed. As far as I can see, there's no way to mux MPEG4 video with MediaMuxer right now while using the public API, without modifying the platform source. Given the issues in Utils.cpp above, you can't mux any video format that requires codec specific data, except for H264. If VP8 is an option, you can mux that into webm files (together with vorbis audio), but hardware encoders for VP8 is probably much less common than hardware encoders for MPEG4. share improve this answer answered  Dec 27 '14 at 18:17 mstorsjo 7,176 2 13 32

This issue has been reported upstream at b.android.com/90138, and a fix has been submitted at android-review.googlesource.com/120945. –  mstorsjo  Dec 27 '14 at 19:24 Thanks for reporting this issue and fixing the same into mainstream tree. –  Ganesh  Jan 5 '15 at 16:23 Well, the fix hasn't been merged yet. I actually found 2 different changes awaiting review that do almost the same, that have been submitted by other people (seemingly unrelated to this post here) but that nobody have merged or even commented on yet. –  mstorsjo  Jan 5 '15 at 16:28 1   My fix at least merges cleanly into their internal tree (according to the buildbot a few days ago), so I don't think they have got any fix for it yet. But often it can take a few months before they react to patches sent via the review system. –  mstorsjo  Jan 5 '15 at 16:42 1   My fix in android-review.googlesource.com/120945 has now been merged in AOSP master, so it will hopefully be part of the next major release. –  mstorsjo  May 20 '15 at 17:08 show 4 more comments

up vote 2 down vote

I presume you have the ability to modify the Stagefright sources and hence, I have a proposed solution for your problem, but one which requires a customization. Background: When an encoder completes encoding, the first buffer will have the csd information which is usually tagged with OMX_BUFFERFLAG_CODECCONFIG flag. When such a buffer is returned to the MediaCodec, it shall store the same as csd-0 in MediaCodec::amendOutputFormatWithCodecSpecificData. Now, when this buffer is given to MediaMuxer, the same is processed as part of addTrack,in which convertMessageToMetadata is invoked. If you refer to the implementation of the same, we can observe that only AVC is handled for video and defaults to audio for ESDS creation. EDIT: Here, my recommendation is to modify this line as below and try your experiment } if (mime.startsWith("audio/") || (!strcmp(mime, MEDIA_MIMETYPE_VIDEO_MPEG4)) { With this change, I feel it should work for MPEG4 video track also. The change is to convert the else if into if as the previous check for video will also try to process the data, but only for AVC. share improve this answer edited Dec 30 '14 at 0:48 answered  Dec 21 '14 at 13:13 Ganesh 5,145 2 21 49

Won't code at line 552: "if (mime.startsWith("video/")) {" get all video buffers? Anyway, I would need to be able to export mp4v-es using MediaCodec public API (so I can manipulate buffer contents as needed) but not resort on jni and native code. –  user1592546  Dec 22 '14 at 17:33 @user1592546..Yes you are right. We can overcome this by removing the else in else if so that there are 2 checks. To optimize this, for the first video check, we could add another part to verify if the MIMEtype is AVC or not. Please see my edited answer above. –  Ganesh  Dec 30 '14 at 0:45  1   I'll accept mstorsjo's answer, for completeness, further reference, and providing the link to full solution. Anyway, this answer provided more insight to the issue, and is appreciated. –  user1592546  Jan 5 '15 at 15:15 @user1592546.. This is fine. I am happy that your problem is solved. –  Ganesh  Jan 5 '15 at 16:24 Well, it still does not work the way I want :) i.e. only java side coding, but I have to accept that. –  user1592546  Jan 8 '15 at 11:22 add a comment