一 FBO
一、FBO的核心概念
FBO（Frame Buffer Object，帧缓冲对象）是OpenGL/WebGL等图形API中用于离屏渲染的核心组件，其本质是显存中可动态管理的缓冲区容器，通过关联纹理或渲染缓冲对象（RBO）实现颜色、深度、模板缓冲区的灵活配置。
-
核心功能：
-
替代默认帧缓冲：默认帧缓冲由窗口系统管理，仅用于屏幕输出；FBO允许开发者创建自定义缓冲区，将渲染结果输出到纹理或RBO中，实现离屏渲染。
-
多目标渲染（MRT）：支持同时绑定多个颜色附件（如
GL_COLOR_ATTACHMENT0、GL_COLOR_ATTACHMENT1），实现单次渲染到多个目标（如颜色、法线、深度等）。 -
动态资源管理：通过挂接纹理或RBO，灵活切换渲染目标，避免显存拷贝，提升性能。
-
-
关键组成：
-
颜色附件：存储颜色数据，支持纹理或RBO挂接。
-
深度/模板附件：通常使用RBO存储深度和模板信息，格式如
GL_DEPTH24_STENCIL8。 -
挂接点：FBO提供多个逻辑挂接点（如
GL_COLOR_ATTACHMENT0），用于绑定实际存储对象。
-
二、FBO在渲染管线中的位置
在图形渲染管线中，FBO位于光栅化与输出合并阶段之后，作为最终的渲染目标容器。具体流程如下：
-
顶点处理：顶点着色器处理几何数据，生成裁剪空间坐标。
-
光栅化：将几何图元转换为片段（Fragment）。
-
片段处理：片段着色器计算颜色、深度等属性，执行深度/模板测试。
-
输出合并：通过FBO将测试通过的片段数据写入颜色、深度、模板缓冲区：
-
默认管线：写入窗口系统提供的默认帧缓冲（屏幕输出）。
-
FBO管线：写入自定义的纹理或RBO（离屏渲染）。
-
-
FBO的绑定与切换：
-
使用
glBindFramebuffer(GL_FRAMEBUFFER, fbo)绑定FBO，后续渲染操作将输出到其附件。 -
通过
glBindFramebuffer(GL_FRAMEBUFFER, 0)切换回默认帧缓冲，恢复屏幕显示。
-
三、FBO与渲染管线的交互细节
-
数据流向：
-
光栅化后的片段数据通过FBO的挂接点（如颜色附件纹理）存储，供后续渲染或后处理使用。
-
例如，将场景渲染到FBO纹理后，可通过全屏四边形进行模糊处理（后处理阶段）。
-
-
性能优势：
-
减少状态切换：复用FBO附件避免频繁绑定不同纹理。
-
内存优化：RBO直接存储深度/模板数据，无需额外内存开销。
-
-
完整性验证：
-
使用
glCheckFramebufferStatus()检查FBO配置是否合法（如附件尺寸一致、格式匹配）。
-
二、FBO 使用流程
1. 创建并绑定FBO
GLuint fbo;
glGenFramebuffers(1, &fbo); // 生成FBO对象
glBindFramebuffer(GL_FRAMEBUFFER, fbo); // 绑定FBO
-
作用：创建独立的渲染目标容器，后续操作将输出到此FBO而非默认帧缓冲。
2. 创建颜色附件（纹理或RBO）
选项1：纹理附件（推荐用于颜色输出）
GLuint colorTex;
glGenTextures(1, &colorTex);
glBindTexture(GL_TEXTURE_2D, colorTex);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, colorTex, 0); // 附加到颜色附件点
-
关键参数：
GL_RGBA格式、GL_LINEAR过滤、GL_UNSIGNED_BYTE数据类型。
选项2：RBO附件（适用于颜色缓冲区优化）
GLuint rboColor;
glGenRenderbuffers(1, &rboColor);
glBindRenderbuffer(GL_RENDERBUFFER, rboColor);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8, width, height);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, rboColor);
3. 创建深度/模板附件（RBO）
GLuint rboDepth;
glGenRenderbuffers(1, &rboDepth);
glBindRenderbuffer(GL_RENDERBUFFER, rboDepth);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, width, height); // 深度+模板
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, rboDepth);
-
作用：存储深度和模板信息，支持深度测试和模板测试。
4. 检查FBO完整性
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
std::cerr << "FBO创建失败！状态码：" << glCheckFramebufferStatus(GL_FRAMEBUFFER) << std::endl;
}
-
必要性：确保附件配置正确（如尺寸一致、格式兼容）。
5. 渲染到FBO
glBindFramebuffer(GL_FRAMEBUFFER, fbo); // 绑定FBO
glViewport(0, 0, width, height); // 设置视口匹配FBO尺寸
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); // 清除缓冲区
// 绘制场景（使用当前绑定的着色器和VAO）
drawScene();
glBindFramebuffer(GL_FRAMEBUFFER, 0); // 解绑，恢复默认帧缓冲
-
视口设置：必须与FBO附件尺寸一致，否则渲染结果可能错乱。
6. 使用FBO渲染结果
// 绑定默认帧缓冲（屏幕）
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glViewport(0, 0, screenWidth, screenHeight); // 恢复屏幕视口
// 使用着色器将FBO纹理绘制到屏幕
shader.use();
glBindTexture(GL_TEXTURE_2D, colorTex); // 绑定FBO颜色纹理
drawQuad(); // 绘制全屏四边形
-
后处理：通过屏幕着色器对FBO纹理应用模糊、HDR等效果。
7. 资源清理
glDeleteFramebuffers(1, &fbo);
glDeleteTextures(1, &colorTex);
glDeleteRenderbuffers(1, &rboDepth);
-
避免内存泄漏：释放不再使用的OpenGL对象。
扩展：多颜色附件（MRT）
// 创建第二个颜色附件
GLuint colorTex2;
glGenTextures(1, &colorTex2);
glBindTexture(GL_TEXTURE_2D, colorTex2);
glTexImage2D(GL_TEXTURE_2D, 0, GL_R32F, width, height, 0, GL_RED, GL_FLOAT, nullptr);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, colorTex2, 0);
// 指定输出到多个颜色缓冲区
GLenum drawBufs[] = { GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1 };
glDrawBuffers(2, drawBufs);
-
应用场景：同时输出颜色、法线、深度等信息到不同纹理。
常见问题与调试
-
黑屏或无输出
-
检查FBO绑定状态和视口尺寸是否匹配。
-
确保至少有一个颜色附件被正确绑定（
GL_COLOR_ATTACHMENT0）。
-
-
深度测试失效
-
确认深度附件已附加（
GL_DEPTH_ATTACHMENT或GL_DEPTH_STENCIL_ATTACHMENT）。 -
启用深度测试：
glEnable(GL_DEPTH_TEST)。
-
-
纹理模糊
-
检查纹理过滤模式（如
GL_NEAREST避免插值）。 -
确保渲染到FBO时视口与纹理尺寸一致。
-
完整代码示例（简化版）
// 初始化FBO
GLuint fbo, colorTex, rboDepth;
glGenFramebuffers(1, &fbo);
glGenTextures(1, &colorTex);
glGenRenderbuffers(1, &rboDepth);
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
// 颜色附件（纹理）
glBindTexture(GL_TEXTURE_2D, colorTex);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 800, 600, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, colorTex, 0);
// 深度附件（RBO）
glBindRenderbuffer(GL_RENDERBUFFER, rboDepth);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, 800, 600);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, rboDepth);
// 检查完整性
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
// 错误处理
}
// 渲染到FBO
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glViewport(0, 0, 800, 600);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
drawScene();
// 解绑并使用结果
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glViewport(0, 0, screenWidth, screenHeight);
shader.use();
glBindTexture(GL_TEXTURE_2D, colorTex);
drawQuad();
三 相机多级滤镜
一 相机预览
package com.example.cameravideofbo
import android.content.Context
import android.content.pm.PackageManager
import android.graphics.SurfaceTexture
import android.hardware.camera2.*
import android.os.Handler
import android.os.HandlerThread
import android.util.Log
import android.view.Surface
import java.io.IOException
class CameraManager(
private val context: Context,
private val surfaceTexture: SurfaceTexture
) {
private val TAG = "CameraManager"
// 相机相关属性
private val cameraManager: android.hardware.camera2.CameraManager
private var cameraDevice: CameraDevice? = null
private var captureSession: CameraCaptureSession? = null
private var backgroundThread: HandlerThread? = null
private var backgroundHandler: Handler? = null
private var cameraPreviewSurface: Surface? = null
// 相机ID
private var cameraId: String = ""
// 构造函数初始化
init {
this.cameraManager = context.getSystemService(Context.CAMERA_SERVICE) as android.hardware.camera2.CameraManager
}
fun startCameraPreview() {
try {
Log.d(TAG, "Starting camera preview")
// 启动后台线程
startBackgroundThread()
// 获取相机ID
cameraId = getCameraId()
// 打开相机
openCamera(cameraId)
} catch (e: Exception) {
Log.e(TAG, "Error starting camera preview: ${e.message}", e)
throw IOException("Failed to start camera preview", e)
}
}
// 获取相机ID（默认使用后置相机）
private fun getCameraId(): String {
try {
val cameraIds = cameraManager.cameraIdList
for (id in cameraIds) {
val characteristics = cameraManager.getCameraCharacteristics(id)
val facing = characteristics.get(CameraCharacteristics.LENS_FACING)
if (facing == CameraCharacteristics.LENS_FACING_BACK) {
return id
}
}
// 如果没有后置相机，返回第一个相机ID
if (cameraIds.isNotEmpty()) {
return cameraIds[0]
}
throw IOException("No camera available")
} catch (e: Exception) {
Log.e(TAG, "Error getting camera ID: ${e.message}", e)
throw IOException("Failed to get camera ID", e)
}
}
// 打开相机
private fun openCamera(cameraId: String) {
try {
Log.d(TAG, "Opening camera: $cameraId")
// 检查相机权限
if (context.checkSelfPermission(android.Manifest.permission.CAMERA)
!= PackageManager.PERMISSION_GRANTED) {
Log.e(TAG, "Camera permission not granted")
throw IOException("Camera permission not granted")
}
// 配置SurfaceTexture
surfaceTexture.setDefaultBufferSize(1280, 720) // 设置合适的预览尺寸
cameraPreviewSurface = Surface(surfaceTexture)
// 打开相机
cameraManager.openCamera(
cameraId,
object : CameraDevice.StateCallback() {
override fun onOpened(camera: CameraDevice) {
Log.d(TAG, "Camera opened successfully")
cameraDevice = camera
// 创建相机预览会话
createCameraPreviewSession()
}
override fun onDisconnected(camera: CameraDevice) {
Log.e(TAG, "Camera disconnected")
camera.close()
cameraDevice = null
}
override fun onError(camera: CameraDevice, error: Int) {
Log.e(TAG, "Camera error: $error")
camera.close()
cameraDevice = null
}
},
backgroundHandler
)
} catch (e: Exception) {
Log.e(TAG, "Error opening camera: ${e.message}", e)
throw IOException("Failed to open camera", e)
}
}
// 创建相机预览会话
private fun createCameraPreviewSession() {
try {
val localCamera = cameraDevice ?: run {
Log.e(TAG, "Camera device is null when creating preview session")
return
}
// 创建预览请求构建器
val captureRequestBuilder = localCamera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
// 设置SurfaceTexture作为相机输出目标
captureRequestBuilder.addTarget(cameraPreviewSurface!!)
// 创建预览请求
val captureRequest = captureRequestBuilder.build()
// 创建相机捕获会话
val localSurface = cameraPreviewSurface
if (localSurface != null) {
localCamera.createCaptureSession(
listOf(localSurface),
object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
if (localCamera != cameraDevice) {
return
}
try {
captureSession = session
// 设置预览请求
session.setRepeatingRequest(captureRequest, null, backgroundHandler)
Log.d(TAG, "Camera preview session configured successfully")
} catch (e: CameraAccessException) {
Log.e(TAG, "Error setting repeating request: ${e.message}", e)
}
}
override fun onConfigureFailed(session: CameraCaptureSession) {
Log.e(TAG, "Failed to configure camera capture session")
}
},
backgroundHandler
)
}
} catch (e: Exception) {
Log.e(TAG, "Error creating camera preview session: ${e.message}", e)
}
}
// 启动后台线程
private fun startBackgroundThread() {
backgroundThread = HandlerThread("CameraBackground")
backgroundThread?.start()
backgroundThread?.looper?.let {
backgroundHandler = Handler(it)
}
}
// 停止后台线程
private fun stopBackgroundThread() {
backgroundThread?.quitSafely()
try {
backgroundThread?.join()
backgroundThread = null
backgroundHandler = null
} catch (e: InterruptedException) {
Log.e(TAG, "Error joining background thread: ${e.message}", e)
}
}
// 关闭相机
private fun closeCamera() {
try {
// 停止相机预览
captureSession?.stopRepeating()
captureSession?.close()
captureSession = null
// 关闭相机设备
cameraDevice?.close()
cameraDevice = null
// 释放Surface
cameraPreviewSurface?.release()
cameraPreviewSurface = null
Log.d(TAG, "Camera closed successfully")
} catch (e: Exception) {
Log.e(TAG, "Error closing camera: ${e.message}", e)
}
}
// 停止相机预览
fun stopCameraPreview() {
try {
Log.d(TAG, "Stopping camera preview")
closeCamera()
} catch (e: Exception) {
Log.e(TAG, "Error stopping camera preview: ${e.message}", e)
}
}
// 释放资源
fun release() {
try {
Log.d(TAG, "Releasing camera manager resources")
// 停止相机预览
closeCamera()
// 停止后台线程
stopBackgroundThread()
Log.d(TAG, "Camera manager resources released successfully")
} catch (e: Exception) {
Log.e(TAG, "Error releasing camera manager resources: ${e.message}", e)
}
}
}
二 灰白滤镜
private fun createFilterProgram() {
// 顶点着色器代码
val vertexShaderCode = """
attribute vec4 aPosition;
attribute vec2 aTexCoord;
varying vec2 vTexCoord;
void main() {
gl_Position = aPosition;
vTexCoord = aTexCoord;
}
"""
// 片段着色器代码 - 灰度滤镜
val fragmentShaderCode = """
precision mediump float;
uniform sampler2D uTexture;
varying vec2 vTexCoord;
void main() {
vec4 color = texture2D(uTexture, vTexCoord);
// 灰度公式: 0.299*R + 0.587*G + 0.114*B
float gray = 0.299 * color.r + 0.587 * color.g + 0.114 * color.b;
gl_FragColor = vec4(gray, gray, gray, color.a);
}
"""
filterProgramId = createShaderProgram(vertexShaderCode, fragmentShaderCode, "filter")
}
三 FBO创建
private fun initFBO(width: Int, height: Int) {
// 如果尺寸没有变化且资源已初始化，不需要重新初始化
if (width == fboWidth && height == fboHeight && fboId > 0 && fboTextureId > 0 && tempTextureId > 0) {
return
}
// 释放旧的资源
releaseFboResources()
fboWidth = width
fboHeight = height
try {
// 生成FBO
val fboIds = IntArray(1)
GLES20.glGenFramebuffers(1, fboIds, 0)
fboId = fboIds[0]
// 生成两个纹理：主纹理和临时纹理
val textureIds = IntArray(2)
GLES20.glGenTextures(2, textureIds, 0)
fboTextureId = textureIds[0]
tempTextureId = textureIds[1]
// 初始化两个纹理
initializeTexture(fboTextureId, width, height)
initializeTexture(tempTextureId, width, height)
// 绑定FBO并附加主纹理
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId)
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_2D, fboTextureId, 0)
// 检查FBO状态
val status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER)
if (status != GLES20.GL_FRAMEBUFFER_COMPLETE) {
Log.e(TAG, "FBO initialization failed: $status")
releaseFboResources()
return
}
// 创建滤镜着色器程序（如果尚未创建）
if (filterProgramId <= 0) {
createFilterProgram()
}
} catch (e: Exception) {
Log.e(TAG, "Error initializing FBO: ${e.message}", e)
releaseFboResources()
} finally {
// 确保解绑
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0)
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0)
}
}
// 初始化单个纹理
private fun initializeTexture(textureId: Int, width: Int, height: Int) {
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId)
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, width, height, 0,
GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
}
四 渲染
override fun onDrawFrame(gl: GL10?) {
// 参数验证
if (surfaceTexture == null || cameraProgramId <= 0 || screenProgramId <= 0 || filterProgramId <= 0) {
Log.w(TAG, "Skipping frame due to missing resources")
return
}
try {
// 第一步：将相机预览渲染到FBO
val localSurfaceTexture = surfaceTexture
localSurfaceTexture!!.updateTexImage()
// 绑定FBO
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId)
// 设置渲染环境
GLES20.glViewport(0, 0, fboWidth, fboHeight)
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f)
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT)
// 使用相机预览着色器程序
GLES20.glUseProgram(cameraProgramId)
// 设置顶点属性
setupVertexAttributes(cameraProgramId)
// 设置MVP矩阵 - 旋转90度修正界面方向
android.opengl.Matrix.setIdentityM(mvpMatrix, 0)
android.opengl.Matrix.rotateM(mvpMatrix, 0, 90.0f, 0.0f, 0.0f, 1.0f)
val uMVPMatrixLocation = GLES20.glGetUniformLocation(cameraProgramId, "uMVPMatrix")
GLES20.glUniformMatrix4fv(uMVPMatrixLocation, 1, false, mvpMatrix, 0)
// 绑定相机纹理
bindTexture(textureId, GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0, cameraProgramId, "uTexture")
// 绘制四边形
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
// 清理顶点属性
disableVertexAttributes(cameraProgramId)
// 第二步：在FBO中应用滤镜处理
renderWithFilter()
// 第三步：将FBO中经过滤镜处理的内容渲染到屏幕
// 解绑FBO，确保渲染目标是默认帧缓冲区
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0)
// 清除默认帧缓冲区
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f)
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT)
// 将FBO内容渲染到屏幕
renderToScreen()
} catch (e: Exception) {
Log.e(TAG, "Error in onDrawFrame: ${e.message}", e)
}
}
完整代码
package com.example.cameravideofbo
import android.content.Context
import android.graphics.SurfaceTexture
import android.opengl.*
import android.util.Log
import java.io.IOException
import javax.microedition.khronos.egl.EGLConfig
import javax.microedition.khronos.opengles.GL10
class CameraRenderer(
private val context: Context,
private val surfaceTextureListener: SurfaceTextureListener? = null
) : GLSurfaceView.Renderer {
// 监听器接口
interface SurfaceTextureListener {
fun onSurfaceTextureAvailable(surfaceTexture: SurfaceTexture?)
}
private val TAG = "CameraRenderer"
// 常量定义
private companion object {
private const val DEFAULT_PREVIEW_WIDTH = 1280
private const val DEFAULT_PREVIEW_HEIGHT = 720
}
// 渲染相关属性
private val mvpMatrix = FloatArray(16)
private val textureTransformMatrix = FloatArray(16)
// 窗口尺寸
private var viewWidth: Int = 0
private var viewHeight: Int = 0
// FBO相关属性
private var fboId: Int = 0
private var fboTextureId: Int = 0 // 主FBO纹理
private var tempTextureId: Int = 0 // 临时纹理，用于滤镜处理
private var fboWidth: Int = 0
private var fboHeight: Int = 0
// 着色器程序
private var cameraProgramId: Int = 0 // 相机预览着色器
private var filterProgramId: Int = 0 // 滤镜着色器
private var screenProgramId: Int = 0 // 屏幕渲染着色器
// 纹理
private var textureId: Int = 0
private var surfaceTexture: SurfaceTexture? = null
// 相机相关属性
private var cameraManager: CameraManager? = null
private var previewWidth: Int = 0
private var previewHeight: Int = 0
// 顶点和纹理坐标
private val vertices = floatArrayOf(
-1.0f, -1.0f, 0.0f, // 左下
1.0f, -1.0f, 0.0f, // 右下
-1.0f, 1.0f, 0.0f, // 左上
1.0f, 1.0f, 0.0f // 右上
)
private val texCoords = floatArrayOf(
0.0f, 1.0f, // 左下
1.0f, 1.0f, // 右下
0.0f, 0.0f, // 左上
1.0f, 0.0f // 右上
)
// 顶点和纹理坐标缓冲区
private val vertexBuffer: java.nio.FloatBuffer
private val texCoordBuffer: java.nio.FloatBuffer
init {
// 初始化顶点缓冲区
vertexBuffer = java.nio.ByteBuffer.allocateDirect(vertices.size * 4)
.order(java.nio.ByteOrder.nativeOrder())
.asFloatBuffer()
vertexBuffer.put(vertices).position(0)
// 初始化纹理坐标缓冲区
texCoordBuffer = java.nio.ByteBuffer.allocateDirect(texCoords.size * 4)
.order(java.nio.ByteOrder.nativeOrder())
.asFloatBuffer()
texCoordBuffer.put(texCoords).position(0)
// 初始化MVP矩阵为单位矩阵
android.opengl.Matrix.setIdentityM(mvpMatrix, 0)
}
// 初始化FBO和滤镜
private fun initFBO(width: Int, height: Int) {
// 如果尺寸没有变化且资源已初始化，不需要重新初始化
if (width == fboWidth && height == fboHeight && fboId > 0 && fboTextureId > 0 && tempTextureId > 0) {
return
}
// 释放旧的资源
releaseFboResources()
fboWidth = width
fboHeight = height
try {
// 生成FBO
val fboIds = IntArray(1)
GLES20.glGenFramebuffers(1, fboIds, 0)
fboId = fboIds[0]
// 生成两个纹理：主纹理和临时纹理
val textureIds = IntArray(2)
GLES20.glGenTextures(2, textureIds, 0)
fboTextureId = textureIds[0]
tempTextureId = textureIds[1]
// 初始化两个纹理
initializeTexture(fboTextureId, width, height)
initializeTexture(tempTextureId, width, height)
// 绑定FBO并附加主纹理
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId)
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_2D, fboTextureId, 0)
// 检查FBO状态
val status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER)
if (status != GLES20.GL_FRAMEBUFFER_COMPLETE) {
Log.e(TAG, "FBO initialization failed: $status")
releaseFboResources()
return
}
// 创建滤镜着色器程序（如果尚未创建）
if (filterProgramId <= 0) {
createFilterProgram()
}
} catch (e: Exception) {
Log.e(TAG, "Error initializing FBO: ${e.message}", e)
releaseFboResources()
} finally {
// 确保解绑
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0)
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0)
}
}
// 初始化单个纹理
private fun initializeTexture(textureId: Int, width: Int, height: Int) {
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId)
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, width, height, 0,
GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
}
// 释放FBO相关资源
private fun releaseFboResources() {
if (tempTextureId > 0) {
GLES20.glDeleteTextures(1, intArrayOf(tempTextureId), 0)
tempTextureId = 0
}
if (fboTextureId > 0) {
GLES20.glDeleteTextures(1, intArrayOf(fboTextureId), 0)
fboTextureId = 0
}
if (fboId > 0) {
GLES20.glDeleteFramebuffers(1, intArrayOf(fboId), 0)
fboId = 0
}
}
// 创建滤镜着色器程序（灰度滤镜）
private fun createFilterProgram() {
// 顶点着色器代码
val vertexShaderCode = """
attribute vec4 aPosition;
attribute vec2 aTexCoord;
varying vec2 vTexCoord;
void main() {
gl_Position = aPosition;
vTexCoord = aTexCoord;
}
"""
// 片段着色器代码 - 灰度滤镜
val fragmentShaderCode = """
precision mediump float;
uniform sampler2D uTexture;
varying vec2 vTexCoord;
void main() {
vec4 color = texture2D(uTexture, vTexCoord);
// 灰度公式: 0.299*R + 0.587*G + 0.114*B
float gray = 0.299 * color.r + 0.587 * color.g + 0.114 * color.b;
gl_FragColor = vec4(gray, gray, gray, color.a);
}
"""
filterProgramId = createShaderProgram(vertexShaderCode, fragmentShaderCode, "filter")
}
// 渲染带有滤镜的帧 - 使用临时纹理避免同时读写
private fun renderWithFilter() {
if (fboId <= 0 || filterProgramId <= 0 || tempTextureId <= 0 || fboTextureId <= 0) return
try {
// 绑定FBO
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId)
GLES20.glViewport(0, 0, fboWidth, fboHeight)
// 第一步：将FBO纹理的内容渲染到临时纹理（应用滤镜）
renderTextureToTexture(fboTextureId, tempTextureId, filterProgramId)
// 第二步：将临时纹理的内容复制回主FBO纹理
renderTextureToTexture(tempTextureId, fboTextureId, screenProgramId)
} catch (e: Exception) {
Log.e(TAG, "Error rendering with filter: ${e.message}", e)
} finally {
// 确保解绑FBO
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0)
}
}
// 将一个纹理渲染到另一个纹理
private fun renderTextureToTexture(sourceTextureId: Int, targetTextureId: Int, programId: Int) {
// 切换FBO的输出纹理
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_2D, targetTextureId, 0)
// 使用指定的着色器程序
GLES20.glUseProgram(programId)
// 获取属性和统一变量的位置
val aPositionLocation = GLES20.glGetAttribLocation(programId, "aPosition")
val aTexCoordLocation = GLES20.glGetAttribLocation(programId, "aTexCoord")
val uTextureLocation = GLES20.glGetUniformLocation(programId, "uTexture")
// 设置顶点属性
vertexBuffer.position(0)
GLES20.glEnableVertexAttribArray(aPositionLocation)
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer)
texCoordBuffer.position(0)
GLES20.glEnableVertexAttribArray(aTexCoordLocation)
GLES20.glVertexAttribPointer(aTexCoordLocation, 2, GLES20.GL_FLOAT, false, 0, texCoordBuffer)
// 设置MVP矩阵（如果着色器程序使用）
val uMVPMatrixLocation = GLES20.glGetUniformLocation(programId, "uMVPMatrix")
if (uMVPMatrixLocation >= 0) {
android.opengl.Matrix.setIdentityM(mvpMatrix, 0)
GLES20.glUniformMatrix4fv(uMVPMatrixLocation, 1, false, mvpMatrix, 0)
}
// 绑定源纹理
GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, sourceTextureId)
GLES20.glUniform1i(uTextureLocation, 0)
// 绘制四边形
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
// 清理状态
GLES20.glDisableVertexAttribArray(aPositionLocation)
GLES20.glDisableVertexAttribArray(aTexCoordLocation)
}
// 将FBO中的内容渲染到屏幕
private fun renderToScreen() {
if (screenProgramId <= 0 || fboTextureId <= 0 || viewWidth <= 0 || viewHeight <= 0) return
try {
// 设置视口为窗口大小
GLES20.glViewport(0, 0, viewWidth, viewHeight)
// 使用屏幕渲染着色器程序
GLES20.glUseProgram(screenProgramId)
// 设置顶点和纹理坐标
setupVertexAttributes(screenProgramId)
// 设置MVP矩阵
android.opengl.Matrix.setIdentityM(mvpMatrix, 0)
val uMVPMatrixLocation = GLES20.glGetUniformLocation(screenProgramId, "uMVPMatrix")
GLES20.glUniformMatrix4fv(uMVPMatrixLocation, 1, false, mvpMatrix, 0)
// 绑定FBO纹理
bindTexture(fboTextureId, GLES20.GL_TEXTURE_2D, 0, screenProgramId, "uTexture")
// 绘制四边形
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
} catch (e: Exception) {
Log.e(TAG, "Error rendering to screen: ${e.message}", e)
} finally {
// 清理顶点属性
disableVertexAttributes(screenProgramId)
}
}
// 设置顶点和纹理坐标属性
private fun setupVertexAttributes(programId: Int) {
val aPositionLocation = GLES20.glGetAttribLocation(programId, "aPosition")
val aTexCoordLocation = GLES20.glGetAttribLocation(programId, "aTexCoord")
vertexBuffer.position(0)
GLES20.glEnableVertexAttribArray(aPositionLocation)
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer)
texCoordBuffer.position(0)
GLES20.glEnableVertexAttribArray(aTexCoordLocation)
GLES20.glVertexAttribPointer(aTexCoordLocation, 2, GLES20.GL_FLOAT, false, 0, texCoordBuffer)
}
// 禁用顶点属性
private fun disableVertexAttributes(programId: Int) {
val aPositionLocation = GLES20.glGetAttribLocation(programId, "aPosition")
val aTexCoordLocation = GLES20.glGetAttribLocation(programId, "aTexCoord")
GLES20.glDisableVertexAttribArray(aPositionLocation)
GLES20.glDisableVertexAttribArray(aTexCoordLocation)
}
// 绑定纹理到指定纹理单元
private fun bindTexture(textureId: Int, textureType: Int, textureUnit: Int, programId: Int, uniformName: String) {
GLES20.glActiveTexture(GLES20.GL_TEXTURE0 + textureUnit)
GLES20.glBindTexture(textureType, textureId)
GLES20.glUniform1i(GLES20.glGetUniformLocation(programId, uniformName), textureUnit)
}
// 启动相机预览
@Throws(IOException::class)
fun startCameraPreview() {
if (surfaceTexture == null) {
throw IOException("SurfaceTexture not available")
}
// 初始化相机管理器
val localSurfaceTexture = surfaceTexture
if (localSurfaceTexture != null) {
cameraManager = CameraManager(context, localSurfaceTexture)
cameraManager?.startCameraPreview()
}
}
// 停止相机预览
fun stopCameraPreview() {
cameraManager?.stopCameraPreview()
}
override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {
try {
Log.d(TAG, "Surface created")
// 初始化OpenGL环境
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f)
GLES20.glDisable(GLES20.GL_DEPTH_TEST)
GLES20.glEnable(GLES20.GL_TEXTURE_2D)
// 创建着色器程序
createCameraProgram()
createScreenProgram()
// 生成相机纹理
generateTexture()
// 创建SurfaceTexture
createSurfaceTexture()
// 验证着色器程序创建成功
if (cameraProgramId <= 0 || screenProgramId <= 0) {
Log.e(TAG, "Failed to create shader programs")
} else {
Log.d(TAG, "All shader programs created successfully")
}
} catch (e: Exception) {
Log.e(TAG, "Error in onSurfaceCreated: ${e.message}", e)
}
}
override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {
try {
Log.d(TAG, "Surface changed: width=$width, height=$height")
// 参数验证
if (width <= 0 || height <= 0) {
Log.w(TAG, "Invalid surface dimensions: $width x $height")
return
}
// 保存窗口宽高
viewWidth = width
viewHeight = height
// 设置视口
GLES20.glViewport(0, 0, width, height)
// 初始化FBO
initFBO(width, height)
} catch (e: Exception) {
Log.e(TAG, "Error in onSurfaceChanged: ${e.message}", e)
}
}
override fun onDrawFrame(gl: GL10?) {
// 参数验证
if (surfaceTexture == null || cameraProgramId <= 0 || screenProgramId <= 0 || filterProgramId <= 0) {
Log.w(TAG, "Skipping frame due to missing resources")
return
}
try {
// 第一步：将相机预览渲染到FBO
val localSurfaceTexture = surfaceTexture
localSurfaceTexture!!.updateTexImage()
// 绑定FBO
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId)
// 设置渲染环境
GLES20.glViewport(0, 0, fboWidth, fboHeight)
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f)
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT)
// 使用相机预览着色器程序
GLES20.glUseProgram(cameraProgramId)
// 设置顶点属性
setupVertexAttributes(cameraProgramId)
// 设置MVP矩阵 - 旋转90度修正界面方向
android.opengl.Matrix.setIdentityM(mvpMatrix, 0)
android.opengl.Matrix.rotateM(mvpMatrix, 0, 90.0f, 0.0f, 0.0f, 1.0f)
val uMVPMatrixLocation = GLES20.glGetUniformLocation(cameraProgramId, "uMVPMatrix")
GLES20.glUniformMatrix4fv(uMVPMatrixLocation, 1, false, mvpMatrix, 0)
// 绑定相机纹理
bindTexture(textureId, GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0, cameraProgramId, "uTexture")
// 绘制四边形
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
// 清理顶点属性
disableVertexAttributes(cameraProgramId)
// 第二步：在FBO中应用滤镜处理
renderWithFilter()
// 第三步：将FBO中经过滤镜处理的内容渲染到屏幕
// 解绑FBO，确保渲染目标是默认帧缓冲区
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0)
// 清除默认帧缓冲区
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f)
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT)
// 将FBO内容渲染到屏幕
renderToScreen()
} catch (e: Exception) {
Log.e(TAG, "Error in onDrawFrame: ${e.message}", e)
}
}
// 创建着色器程序
private fun createShaderProgram(vertexCode: String, fragmentCode: String, programName: String): Int {
try {
// 编译着色器
val vertexShaderId = compileShader(GLES20.GL_VERTEX_SHADER, vertexCode)
val fragmentShaderId = compileShader(GLES20.GL_FRAGMENT_SHADER, fragmentCode)
if (vertexShaderId <= 0 || fragmentShaderId <= 0) {
return 0
}
// 链接程序
val programId = GLES20.glCreateProgram()
GLES20.glAttachShader(programId, vertexShaderId)
GLES20.glAttachShader(programId, fragmentShaderId)
GLES20.glLinkProgram(programId)
// 检查链接状态
val linkStatus = IntArray(1)
GLES20.glGetProgramiv(programId, GLES20.GL_LINK_STATUS, linkStatus, 0)
if (linkStatus[0] != GLES20.GL_TRUE) {
val error = GLES20.glGetProgramInfoLog(programId)
Log.e(TAG, "Error linking $programName program: $error")
GLES20.glDeleteProgram(programId)
return 0
}
return programId
} catch (e: Exception) {
Log.e(TAG, "Error creating $programName program: ${e.message}", e)
return 0
}
}
// 编译单个着色器
private fun compileShader(type: Int, shaderCode: String): Int {
val shaderId = GLES20.glCreateShader(type)
GLES20.glShaderSource(shaderId, shaderCode)
GLES20.glCompileShader(shaderId)
// 检查编译状态
val compileStatus = IntArray(1)
GLES20.glGetShaderiv(shaderId, GLES20.GL_COMPILE_STATUS, compileStatus, 0)
if (compileStatus[0] != GLES20.GL_TRUE) {
val error = GLES20.glGetShaderInfoLog(shaderId)
val shaderType = if (type == GLES20.GL_VERTEX_SHADER) "vertex" else "fragment"
Log.e(TAG, "Error compiling $shaderType shader: $error")
GLES20.glDeleteShader(shaderId)
return 0
}
return shaderId
}
// 创建相机预览着色器程序
private fun createCameraProgram() {
// 顶点着色器代码
val vertexShaderCode = """
attribute vec4 aPosition;
attribute vec2 aTexCoord;
uniform mat4 uMVPMatrix;
varying vec2 vTexCoord;
void main() {
gl_Position = uMVPMatrix * aPosition;
vTexCoord = aTexCoord;
}
"""
// 片段着色器代码 - 用于相机预览（OES纹理）
val fragmentShaderCode = """
#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES uTexture;
varying vec2 vTexCoord;
void main() {
gl_FragColor = texture2D(uTexture, vTexCoord);
}
"""
cameraProgramId = createShaderProgram(vertexShaderCode, fragmentShaderCode, "camera")
}
// 创建屏幕渲染着色器程序
private fun createScreenProgram() {
// 顶点着色器代码
val vertexShaderCode = """
attribute vec4 aPosition;
attribute vec2 aTexCoord;
uniform mat4 uMVPMatrix;
varying vec2 vTexCoord;
void main() {
gl_Position = uMVPMatrix * aPosition;
vTexCoord = aTexCoord;
}
"""
// 片段着色器代码 - 支持普通2D纹理
val fragmentShaderCode = """
precision mediump float;
uniform sampler2D uTexture;
varying vec2 vTexCoord;
void main() {
gl_FragColor = texture2D(uTexture, vTexCoord);
}
"""
screenProgramId = createShaderProgram(vertexShaderCode, fragmentShaderCode, "screen")
}
private fun generateTexture() {
try {
Log.d(TAG, "Generating camera texture")
// 生成纹理
val textureIds = IntArray(1)
GLES20.glGenTextures(1, textureIds, 0)
textureId = textureIds[0]
if (textureId <= 0) {
Log.e(TAG, "Failed to generate camera texture")
return
}
// 设置纹理参数
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId)
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
Log.d(TAG, "Camera texture generated successfully: $textureId")
} catch (e: Exception) {
Log.e(TAG, "Error generating camera texture: ${e.message}", e)
} finally {
// 解绑纹理
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0)
}
}
private fun createSurfaceTexture() {
// 创建SurfaceTexture
surfaceTexture = SurfaceTexture(textureId)
surfaceTexture?.setOnFrameAvailableListener {
// 当有新帧可用时，可以在这里进行处理
}
// 通知监听器SurfaceTexture已创建
surfaceTextureListener?.onSurfaceTextureAvailable(surfaceTexture)
}
fun setPreviewSize(width: Int, height: Int) {
previewWidth = width
previewHeight = height
}
fun release() {
try {
Log.d(TAG, "Releasing renderer resources")
// 停止相机预览并释放CameraManager资源
cameraManager?.release()
cameraManager = null
// 释放FBO相关资源
releaseFboResources()
// 释放相机纹理
if (textureId > 0) {
GLES20.glDeleteTextures(1, intArrayOf(textureId), 0)
textureId = 0
}
// 释放着色器程序
if (cameraProgramId > 0) {
GLES20.glDeleteProgram(cameraProgramId)
cameraProgramId = 0
}
if (screenProgramId > 0) {
GLES20.glDeleteProgram(screenProgramId)
screenProgramId = 0
}
if (filterProgramId > 0) {
GLES20.glDeleteProgram(filterProgramId)
filterProgramId = 0
}
// 释放SurfaceTexture
surfaceTexture?.release()
surfaceTexture = null
Log.d(TAG, "Renderer resources released successfully")
} catch (e: Exception) {
Log.e(TAG, "Error releasing renderer resources: ${e.message}", e)
}
}
}
扫一扫
2006
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
