AAC ADTS音频解码之路

本帖最后由 jingjin221 于 2016-3-16 11:51 编辑

我这里有一份从TS流中解复用过后的AAC码流,用OMXCODEC来解码始终解码不出来,请大家帮忙验证一下,红色部分为错误打印信息E/OMXCodec(6058): [omx.google.aac.decoder] ERROR(0x80001001, 8195)

经过调试发现,是在SoftAAC2::onQueueFilled aacDecoder_ConfigRaw 出问题了导致如上问题!
参考 Android平台aac谷歌软解框架和流程、解码库学习_LLYF_新浪博客 这个帖子得知在向AAC解码器送真实数据前会调用aacDecoder_ConfigRaw 去重新获取aac的一些信息,如sampleRate和numChannels,保存在CStreamInfo结构体中。而我是直接送的音频数据包括ADTS头部
但是要送的这个起始数据究竟是如何得到的呢?我尝试播放M4A的歌曲来查看这个起始数据,发现起始数据一般是2个字节,但是具体内容就不同了!求大神分析!我尝试自己伪造起始数据,aacDecoder_ConfigRaw通过了,但是解码的时候还是失败了啊!!!
test.rar (435 KB)

本帖最后由 jingjin221 于 2016-3-16 11:53 编辑

经过半天的调试,AAC解码已经正常结贴了吧!做如下总结,也可以希望以后的同学们少走弯路
1.我起初在OMXCODEC下调试,MP3解码没有问题,AAC始终不行,调试发现是由于少送了CSD信息,另外需要设置关键配置meta->setInt32(kKeyIsADTS, 1);于是自己根据ADTS HEADER构造了CSD信息,但是送入解码器总是出现奇奇怪怪的问题。实在无法解释,于是放弃OMXCODEC,利用ACODEC来解码
2.利用ACODEC解码在JAVA层也就是MEDIACODEC,需要配置信息如下
[mw_shl_code=c,false] if(audio_cfg.stream_type == AUDIO_MP3)
{
mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
format->setString(“mime”, mimetype);
s->mSampleRate = audio_cfg.sampling_frequency;
}
else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
{
mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
format->setString(“mime”, mimetype);
s->mSampleRate = audio_cfg.sampling_frequency/2;
format->setInt32(“is-adts”, 1);
format->setInt32(“aac-profile”, 0x0002);
}

    channel_configuration = audio_cfg.channel_configuration;

    format->setInt32("sample-rate", s->mSampleRate);
    format->setInt32("channel-count", channel_configuration);[/mw_shl_code]

当然还需自己构造CSD头部,参考如下ADTS解析函数
[mw_shl_code=c,false]static int AAC_ADTS_header_parse(uint8_t *p_data)
{
uint8_t mpeg_version, layer, profile, sampling_frequency_index, channel_configuration;
uint16_t frame_length;

if(!((p_data[0] == 0xFF) && ((p_data[1] & 0xF0) == 0xF0)))      //ADTS syncword all 12 bits must be 1
    return -1;
para.audio.stream_type = AUDIO_AAC_ADTS;

/*
0 for MPEG-4
1 for MPEG-2
*/
mpeg_version = (p_data[1]&0x08) >> 3;
printf("mpeg_version is %d\n", mpeg_version);

/*
0: Main profile
1: Low Complexity profile (LC)
2: Scalable Sampling Rate profile (SSR)
3: (reserved)
*/
profile = (p_data[2]&0xC0)>>6;
printf("profile is %d\n", profile);
para.audio.profile = profile;

/*
There are 13 supported frequencies:
0: 96000 Hz
1: 88200 Hz
2: 64000 Hz
3: 48000 Hz
4: 44100 Hz
5: 32000 Hz
6: 24000 Hz
7: 22050 Hz
8: 16000 Hz
9: 12000 Hz
10: 11025 Hz
11: 8000 Hz
12: 7350 Hz
13: Reserved
14: Reserved
15: frequency is written explictly
*/
unsigned int Sampling_Frequencies[13] = {96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350};
sampling_frequency_index = (p_data[2]&0x3C) >> 2;   //check the table Sampling Frequencies
printf("sampling_frequency_index is %d\n", sampling_frequency_index);
para.audio.sampling_frequency = Sampling_Frequencies[sampling_frequency_index];

/**
0: Defined in AOT Specifc Config
1: 1 channel: front-center
2: 2 channels: front-left, front-right
3: 3 channels: front-center, front-left, front-right
4: 4 channels: front-center, front-left, front-right, back-center
5: 5 channels: front-center, front-left, front-right, back-left, back-right
6: 6 channels: front-center, front-left, front-right, back-left, back-right, LFE-channel
7: 8 channels: front-center, front-left, front-right, side-left, side-right, back-left, back-right, LFE-channel
8-15: Reserved
**/
channel_configuration = ((p_data[2]&0x01) << 1) | ((p_data[3]&0xC0)>>6);
printf("channel_configuration is %d\n", channel_configuration);
para.audio.channel_configuration = channel_configuration;

para.audio.csd[0] = (profile << 4) | (sampling_frequency_index >> 1);
para.audio.csd[1] = ((sampling_frequency_index & 0x01) << 7) | (channel_configuration << 3);
printf("csd-0[0x%x][0x%x]\n", para.audio.csd[0], para.audio.csd[1]);

return 0;

}[/mw_shl_code]
3.不知道为什么在AAC下,明明解析出来的采样率是48K,送进解码器出来的声音明显频率高了,我只要在初始化的时候进行了分频操作!
4.ACODEC比OMXCODEC靠谱多了,也许是该死是READ造成的吧!5.在送数据的时候其实并不用剥去ADTS头部
6.靠谱的ANDROID论坛还是stackoverflow.com.国内怎么就出不了这种网站呢?

再附上音频解码的核心代码吧![mw_shl_code=c,false]static int audio_decoder_init(StagefrightContext *s, struct audio_config audio_cfg){
#ifdef AUDIO_DECODER
//#define PCM_FILE_PLAY_DEBUG
#ifdef PCM_FILE_PLAY_DEBUG
audio_pcm_play(s);
return 0;
#endif

int ret = 0;
sp<MetaData> meta;
const char* mimetype;
int32_t channel_configuration;

printf("%s_%d\n", __FUNCTION__,  __LINE__);

#if (defined OMXCODEC)
meta = new MetaData;
if (meta == NULL) {
printf(“cannot allocate MetaData”);
return -1;
}

if(audio_cfg.stream_type == AUDIO_MP3)
{
	mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
	meta->setCString(kKeyMIMEType, mimetype);
}
else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
{
	mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
	meta->setCString(kKeyMIMEType, mimetype);
	meta->setInt32(kKeyIsADTS, 1);
	meta->setInt32(kKeyAACProfile, 0x0002);
}

s->mSampleRate = audio_cfg.sampling_frequency;
channel_configuration = audio_cfg.channel_configuration;

meta->setInt32(kKeySampleRate, s->mSampleRate);
meta->setInt32(kKeyChannelCount, channel_configuration);

s->mAudioSource = new sp<MediaSource>();
*s->mAudioSource  = new CStageFrightAudioSource(s, meta);

if (s->mAudioSource == NULL) {
	s->mAudioSource = NULL;
	printf("Cannot obtain source / mClient");
	return -1;
}

if (s->mClient.connect() !=  OK) {
	printf("Cannot connect OMX mClient\n");
	ret = -1;
	goto fail;
	}

s->mAudioDecoder= new sp<MediaSource>();
printf("[%s]@OMXCodec::Create____________________________START\n", __FUNCTION__);
*s->mAudioDecoder = OMXCodec::Create(s->mClient.interface(),
	meta,
	false,
	*s->mAudioSource,
	NULL,
	OMXCodec::kSoftwareCodecsOnly,
	NULL);
if (!(s->mAudioDecoder != NULL && (*s->mAudioDecoder)->start() ==  OK)) {
	printf("[%s]@Cannot start decoder\n", __FUNCTION__);
	ret = -1;
	s->mClient.disconnect();
	s->mAudioSource = NULL;
	s->mAudioDecoder = NULL;
	goto fail;
}
printf("[%s]@OMXCodec::Create____________________________END\n", __FUNCTION__);

fail:
return ret;

#elif (defined ACODEC)
sp format;
format = new AMessage;
if(format == NULL) {
printf(“cannot allocate format\n”);
return -1;
}

if(audio_cfg.stream_type == AUDIO_MP3)
{
	mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
	format->setString("mime", mimetype);
	s->mSampleRate = audio_cfg.sampling_frequency;
}
else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
{
	mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
	format->setString("mime", mimetype);
	s->mSampleRate = audio_cfg.sampling_frequency/2;
	format->setInt32("is-adts", 1);
	format->setInt32("aac-profile", 0x0002);
}

channel_configuration = audio_cfg.channel_configuration;

format->setInt32("sample-rate", s->mSampleRate);
format->setInt32("channel-count", channel_configuration);

printf("[%s]@ACodec::Create____________________________START\n", __FUNCTION__);

sp<ALooper> mLooper = new ALooper;
mLooper->setName("MediaCodec_Adio_looper");
mLooper->start(
        false,      // runOnCallingThread
        false,       // canCallJava
        PRIORITY_FOREGROUND);
s->mACodecAudioDecoder = MediaCodec::CreateByType(
            mLooper, mimetype, false /* encoder */);
if(s->mACodecAudioDecoder == NULL)
{
	printf("Failed to create mACodecAudioDecoder\n");
	return -1;
}

ret = s->mACodecAudioDecoder->configure(
            format, NULL /* surface */,
            NULL /* crypto */,
            0 /* flags */);
if(ret != OK)
{
	printf("Failed to configure mACodecAudioDecoder\n");
	return -1;
}

printf("[%s]@ACodec::Create____________________________END\n", __FUNCTION__);

ret  = s->mACodecAudioDecoder->start();
if(ret != OK)
{
	printf("Failed to start mACodecAudioDecoder\n");
	return -1;
}

ret = s->mACodecAudioDecoder->getInputBuffers(&s->mAudioInBuffers);
if(ret != OK)
{
	printf("Failed to getInputBuffers mACodecAudioDecoder\n");
	return -1;
}

ret = s->mACodecAudioDecoder->getOutputBuffers(&s->mAudioOutBuffers);
if(ret != OK)
{
	printf("Failed to getOutputBuffers mACodecAudioDecoder\n");
	return -1;
}

printf("got %d input and %d output buffers", s->mAudioInBuffers.size(), s->mAudioOutBuffers.size());

fail:
return ret;

#endif

#endif
}
[/mw_shl_code]
[mw_shl_code=c,false]static void* audio_decode_sound_thread(void *arg)
{
status_t err;
StagefrightContext s = (StagefrightContext)arg;
MediaBuffer *buffer = NULL;
printf(“[%s]Thread id:%d/n”, FUNCTION, gettid());

if(audio_decoder_init(s, para.audio) == -1)
	return NULL;

size_t frameCount = 0;
if (AudioTrack::getMinFrameCount(&frameCount, AUDIO_STREAM_DEFAULT, s->mSampleRate) != NO_ERROR) {
    return NULL;
}

int nbChannels = 2;
int audioFormat = ENCODING_PCM_16BIT;
size_t size =  frameCount * nbChannels * (audioFormat == ENCODING_PCM_16BIT ? 2 : 1);
printf("size is %d, s->mSampleRate is %d\n", size, s->mSampleRate);

s->mAudioTrack = new AudioTrack(AUDIO_STREAM_MUSIC,
								s->mSampleRate,
								AUDIO_FORMAT_PCM_16_BIT,
								AUDIO_CHANNEL_OUT_STEREO,
								0,
								AUDIO_OUTPUT_FLAG_NONE,
								NULL,
								NULL,
								0);
if ((err = s->mAudioTrack->initCheck()) != OK) {
	printf("AudioTrack initCheck failed\n");
	s->mAudioTrack.clear();
}
s->mAudioTrack->setVolume(1.0f);
s->mAudioTrack->start();

#if 1
#if (defined OMXCODEC)
while(1)
{
status_t status = (*s->mAudioDecoder)->read(&buffer, NULL);

	if (status == OK) {
		printf("%s@AUDIO DECODER OK\n", __FUNCTION__);
		if (buffer->range_length() == 0)
		{
			printf("%s:ERROR_BUFFER_TOO_SMALL\n", __FUNCTION__);
			status = ERROR_BUFFER_TOO_SMALL;
			buffer->release();
			buffer = NULL;
			continue;
		}
		//printf("BUFFER RANGE LENGTH[%d]\n", buffer->range_length());
	}
	else
		;//printf("%s@AUDIO DECODER NOT OK\n", __FUNCTION__);

	if(status == OK) {
		sp<MetaData> outFormat = (*s->mAudioDecoder)->getFormat();
		outFormat->findInt32(kKeySampleRate, &s->mSampleRate);
		printf("SAMPLERATE[%d]\n", s->mSampleRate);
	}

	if (status == OK) {
		s->mAudioTrack->write(buffer->data(), buffer->range_length());
		buffer->release();
		buffer = NULL;
	}

}

#elif (defined ACODEC)
static int first_flag = true;
int sampleSize;
static int64_t kTimeout_audio = 10000;
size_t inIndex;
size_t outIndex;
size_t offset;
size_t len;
int64_t presentationTimeUs;
uint32_t flags;

while(1)
{
	err = s->mACodecAudioDecoder->dequeueInputBuffer(&inIndex, kTimeout_audio);
	if (err == OK) {
		//printf("filling input buffer %d\n", inIndex);

		const sp<ABuffer> &buffer = s->mAudioInBuffers.itemAt(inIndex);

if((para.audio.stream_type == AUDIO_AAC_ADTS) && first_flag)
{
memcpy((uint8_t *)buffer->data(), para.audio.csd, 2);
sampleSize = 2;
first_flag = false;
}
else
{
sampleSize = audio_read_one_frame((uint8_t *)buffer->data());
}
presentationTimeUs = 0;
if(sampleSize <= 0)
break;

		if (buffer->capacity() < sampleSize) {
			printf("buffer capacity overflow\n");
			break;
		}

		buffer->setRange(0, sampleSize);

		err = s->mACodecAudioDecoder->queueInputBuffer(
			inIndex,
			0 /* offset */,
			buffer->size(),
			presentationTimeUs,
			0 /* flag*/);
		//printf("queueInputBuffer err is %d\n", err);
	}

	err = s->mACodecAudioDecoder->dequeueOutputBuffer(&outIndex, &offset, &len, &presentationTimeUs, &flags, kTimeout_audio);
	//printf("dequeueOutputBuffer err is %d\n", err);
	if (err == OK) {
			s->mACodecAudioDecoder->getOutputBuffers(&s->mAudioOutBuffers);
			//printf("got %d output buffers", s->mAudioOutBuffers.size());
			const sp<ABuffer> &buffer = s->mAudioOutBuffers.itemAt(outIndex);
			//printf("output buffers[%d] size[%d]\n",outIndex, buffer->size());
			s->mAudioTrack->write(buffer->data(), buffer->size());
			s->mACodecAudioDecoder->releaseOutputBuffer(outIndex);
	}
}

#endif
#endif
}[/mw_shl_code]

技术贴,顶起来~

你好,我也遇到同样的问题,可以请教一下吗。我的QQ:2273431063

2273431063 发表于 2016-4-11 20:18
你好,我也遇到同样的问题,可以请教一下吗。我的QQ:2273431063

加我QQ512975979

你好
你能告诉我哪些文件萤火虫SDK改变 - https://bitbucket.org/T-Firefly/firenow-lollipop/src修复AAC ADTS?
然后,可以在标记红色部分改变或增加

Hi
Can you tell me which files to change in Firefly SDK - https://bitbucket.org/T-Firefly/firenow-lollipop/src to fix AAC ADTS?
Then can mark parts in red to change or add.

dewettie 发表于 2016-4-22 11:28
你好
你能告诉我哪些文件萤火虫SDK改变 - https://bitbucket.org/T-Firefly/firenow-lollipop/src修复AAC

你的软硬件平台是什么?

jingjin221 发表于 2016-4-22 11:52
你的软硬件平台是什么?

何晶晶221

萤火虫RK3288 - 棒棒堂5.1 - Firenow SDK
HPH RK3288电视盒 - 棒棒堂5.1 - Firenow SDK
下面是我使用SDK的源代码的框架文件夹 - https://bitbucket.org/T-Firefly/firenow-lollipop/src/1c1093c3d5688f102b1847793bccf265be39738e/frameworks?at=Firefly-RK3288.请告诉哪些文件要修改或添加为您修复。

Firefly RK3288 - Lollipop 5.1 - Firenow SDK
HPH RK3288 TV Box - Lollipop 5.1 - Firenow SDK
Here is the frameworks folder of the source code of the SDK I use - https://bitbucket.org/T-Firefly/firenow-lollipop/src/1c1093c3d5688f102b1847793bccf265be39738e/frameworks?at=Firefly-RK3288. Please tell which files to modify or add for your fix.

支持

能否请您分享萤火虫RK3288平台棒棒糖ADTS AAC贴片/变化?
Can you please share ADTS AAC patch/changes for Firefly RK3288 Lollipop platform?

dewettie 发表于 2016-4-27 01:11
能否请您分享萤火虫RK3288平台棒棒糖ADTS AAC贴片/变化?
Can you please share ADTS AAC patch/changes f …

AV FRAMEWORK 我没有更改,参考的rk3288 android4.4的SDK。
本贴只是描述了我在调用AAC解码时所遇到的一些问题!