Movie Gen: A Cast of Media Foundation Models

本文是LLM系列文章,针对《Movie Gen: A Cast of Media Foundation Models》的翻译。

摘要

我们介绍Movie Gen,这是一系列基础模型,可以生成具有不同宽高比和同步音频的高质量1080p高清视频。我们还展示了其他功能,例如基于精确指令的视频编辑和基于用户图像的个性化视频生成。我们的模型在多个任务上设定了最新的技术水平:文本到视频合成、视频个性化、视频编辑、视频到音频生成和文本到音频生成。我们最大的视频生成模型是一个30B参数transformer,其最大上下文长度为73K视频token,对应于以每秒16帧的速度生成16秒的视频。我们在架构、潜在空间、训练目标和配方、数据管理、评估协议、并行化技术和推理优化方面展示了多项技术创新和简化,使我们能够从扩展预训练数据、模型大小和训练计算中获益,以训练大规模媒体生成模型。我们希望本文能帮助研究界加快媒体生成模型的进步和创新。
本文的所有视频均在https://go.fb.me/MovieGenResearchVideos可用。

1 引言

2 概述

3 联合图像和视频生成

4 视频个性化

5 指导式精确视频剪辑

6 联合音效和音乐生成

7 相关工作

8 结论

船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

实现飞书机器人推送消息到指定群组或者用户

飞书是一个集成了聊天、协作和办公功能的企业级平台,广泛应用于企业内部沟通和协作。飞书机器人(Feishu Bot)是一种强大的工具,允许开发者通过编程接口与飞书进行互动,从而实现自动化消息推送和任务管理。本文将详细介绍如何通过飞书机器人推送消息给指定的人或者群组,帮助您快速上手。

qq_41898196的博客 5872

AD服务器搭建

一、角色安装1)打开服务器管理器,添加 角色和功能;2) 点 下一步;3)点 下一步;4)点 下一步;5)点击安装;6-1)将此服务器提升为域控制器;6-2)点下一步;6)输入密码 点 下一步;7)自动生成LIXF,点击下一步;8)确认目录,点下一步;8)确认安装条件,点安装; 8)安装成功,点关闭;...

潇峰的博客 6454

清华电子系保研面试全记录:五轮硬核考核+工程硕博专项解析(附真题)

本文详细解析了清华大学电子工程系保研面试的完整流程,重点剖析了其标志性的五轮专业面试(涵盖电路、电磁场、信号系统、计算机与AI)和综合组群面,并深入解读了工程硕博改革专项的考核要点与备战策略。文章提供了各专业组的真题示例与应对方法,旨在帮助电子信息相关专业学子系统准备,从容应对这场高强度的知识与综合素质挑战。

u0v1w2x3的博客 589

Ubuntu 20.04下IgH EtherCAT主站部署实战

本文详细介绍了在Ubuntu 20.04系统上部署IgH EtherCAT主站的完整流程,包括环境准备、源码编译、内核模块配置、网络接口设置及实时性优化等关键步骤。通过实战案例和常见问题排查,帮助开发者快速掌握EtherCAT主站部署技巧,提升工业自动化系统的稳定性和实时性。

weixin_30596343的博客 1071

告别混乱!AutoSar BswM模式管理实战:手把手教你配置Rule与ActionList(附避坑指南)

本文详细解析了AutoSar BswM模式管理的实战配置,重点介绍了Rule与ActionList的高效组合方法,帮助开发者避免常见陷阱。通过具体案例演示了从基础到复杂的配置步骤,包括模式仲裁、逻辑表达式构建及ActionList优化策略,提升汽车电子系统的协调性与可靠性。

weixin_27066963的博客 260

告别定位不准!Inner-IoU助力YOLOv11边界框回归精度飙升

Inner-IoU 凭借辅助边框的创新设计,成功解决了传统 IoU 损失在小目标、遮挡场景下的收敛难题,为 YOLOv11 注入了更强的定位能力。从原理剖析到代码集成,本教程已为你铺就了一条“精度提升”的实战路径。现在,不妨亲自上手尝试,让你的目标检测模型在各类数据集上都能交出一份“精度优异”的答卷——相信我,当你看到检测框精准贴合目标的那一刻,你会明白这份尝试的价值。

FJN110的博客 192

【论文汇总】Diffusion Models视频生成/视频编辑/可控视频生成/跨模态视频生成

本文总结了Diffusion Models视频生成领域相关的工作,目前共收录142篇,持续更新中。

沉迷单车的追风少年 1674

C# + NAudio:构建支持实时音频处理的语音聊天应用(含回声消除)

本文介绍了如何利用C#和NAudio库开发实时语音聊天应用,重点实现回声消除功能。首先概述了NAudio库在音频处理中的作用,包括音频录制、播放和实时处理能力。然后详细讲解了构建基础语音聊天应用的过程,包括麦克风录制和扬声器播放的实现。接着深入探讨了回声消除的原理和实现方法,提出通过比较麦克风输入和扬声器输出的音频信号来消除回声,并给出了核心代码示例。最后,文章还讨论了性能优化方向,如多线程处理和网络传输扩展。该方案为开发高质量实时语音应用提供了实用参考。

这里只有干货:从 Modbus 通信到 MES 对接,从 YOLO 训练到工控机部署,带你搞定工业软件开发全流程。 1013

迈瑞BS系列全自动生化分析仪LIS协议接口手册(内含HL7和ASTM)

迈瑞BS系列全自动生化分析仪LIS协议接口手册,内含HL7和ASTM)

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

pscad IEEE标准系统搭建说明

pscad IEEE标准系统搭建说明,包含典型的IEEE 9节点、14节点、30节点、39节点和118节点系统。内含标准测试系统的原始数据和潮流结果。

win10 跳过U8环境检测的IE WebControls.rar

win10 安装u8的时候 提示IE WebControls 不通过,这个工具可以欺骗u8的检查。 跳过U8环境检测的IE WebControls。提示导入失败后,重新检测通过。

上一篇: Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement
下一篇: WEB AGENTS WITH WORLD MODELS: LEARNING AND LEVERAGING ENVIRONMENT DYNAMICS IN WEB NAVIGATION
UnknownBody
博客等级 码龄16年 5601粉丝 4744原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值