人工智能 | ShowMeAI资讯日报 #2022.06.29

【最详细公式推导:表 1】Elucidating the Design Space of Diffusion-Based Generative Models 【论文精读】 【公式推导表 1】Elucidating the Design Space of Diffusion-Based Generative Models 【论文精读】 阅读详情

ShowMeAI日报系列全新升级!覆盖AI人工智能 工具&框架 | 项目&代码 | 博文&分享 | 数据&资源 | 研究&论文 等方向。点击查看 历史文章列表,在公众号内订阅话题 #ShowMeAI资讯日报,可接收每日最新推送。点击 专题合辑&电子月刊 快速浏览各专题全集。点击 这里 回复关键字 日报 免费获取AI电子月刊与资料包。

1.工具&框架

预训练模型:YaLM 100B - 100B参数预训练语言模型

tags: [预训练,语言模型,大模型]

大型模型用于文本生成和处理的类GPT开源预训练神经网络

'YaLM 100B - Pretrained language model with 100B parameters,a GPT-like neural network for generating and processing text’ by Yandex

GitHub: https://github.com/yandex/YaLM-100B

工具框架:NFShmServer - 用C++开发的共享内存的插件开发框架

tags: [内存共享,插件]

轻量级、敏捷型、弹性的、分布式支持,让你更快更简单的开发服务端应用

GitHub: https://github.com/yigao/NFShmServer

工具:Nocturne - 数据驱动的快速2D多智能体驾驶模拟器

tags: [多智能体,驾驶模拟,模拟器]

‘Nocturne - A data-driven, fast driving simulator for multi-agent coordination under partial observability.’ by Meta Research

GitHub: https://github.com/facebookresearch/nocturne

工具框架:FidelityFX Super Resolution(FSR) - FidelityFX超分辨率框架

tags: [超分辨率,工具]

‘FidelityFX Super Resolution 2.0.1 (FSR 2.0) - FidelityFX Super Resolution 2’ by GPUOpen Effects

GitHub: https://github.com/GPUOpen-Effects/FidelityFX-FSR2

工具库:NNCF - 神经网络压缩框架

tags: [神经网络,网络压缩]

‘Neural Network Compression Framework (NNCF) - Neural Network Compression Framework for enhanced OpenVINO inference’ by OpenVINO Toolkit

GitHub: https://github.com/openvinotoolkit/nncf

2.项目&代码

代码实现:Recommend-System-TF2.0 - 推荐算法实现

‘于记录在学习推荐系统过程中的知识产出,主要是对经典推荐算法的原理解析及代码实现’ by jc_Lee

GitHub: https://github.com/jc-LeeHub/Recommend-System-tf2.0

3.博文&分享

博文:Transformer前向传播能有多快?

《How fast can we perform a forward pass?》by Jacob Steinhardt

Link: https://bounded-regret.ghost.io/how-fast-can-we-perform-a-forward-pass/

4.数据&资源

资源列表:Rolling-shutter-Effects - 图像畸变校正(滚动快门/径向失真/文字失真等)相关文献资源列表

tags: [图像畸变,图像校正,资源列表]

‘Rolling-shutter-Effects - A curated list of resources on handling Rolling Shutter effects and Radial Distortions’ by Subeesh Vasu

GitHub: https://github.com/subeeshvasu/Awesome-Image-Distortion-Correction

资源列表:ADGC - 深度图聚类相关文献资源大列表

‘ADGC: Awesome Deep Graph Clustering - Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods (papers, codes, and datasets).’ by yueliu1999

GitHub: https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering

5.研究&论文

可以点击 这里 回复关键字 日报,免费获取整理好的6月论文合辑。

论文:Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

论文标题:Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

论文时间:22 Jun 2022

所属领域:计算机视觉,自然语言处理

对应任务:Image Generation,Machine Translation,Text to image generation,Text-to-Image Generation,图像生成,机器翻译,文本到图像生成

论文地址:https://arxiv.org/abs/2206.10789

代码实现:https://github.com/lucidrains/parti-pytorch

论文作者:Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, ZiRui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, Yonghui Wu

论文简介:We present the Pathways Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge./我们提出了Pathways Autoregressive Text-to-Image (Parti)模型,它能生成高保真的逼真图像,并支持涉及复杂构图和世界知识的内容丰富的合成。

论文摘要:We present the Pathways Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge. Parti treats text-to-image generation as a sequence-to-sequence modeling problem, akin to machine translation, with sequences of image tokens as the target outputs rather than text tokens in another language. This strategy can naturally tap into the rich body of prior work on large language models, which have seen continued advances in capabilities and performance through scaling data and model sizes. Our approach is simple: First, Parti uses a Transformer-based image tokenizer, ViT-VQGAN, to encode images as sequences of discrete tokens. Second, we achieve consistent quality improvements by scaling the encoder-decoder Transformer model up to 20B parameters, with a new state-of-the-art zero-shot FID score of 7.23 and finetuned FID score of 3.22 on MS-COCO. Our detailed analysis on Localized Narratives as well as PartiPrompts (P2), a new holistic benchmark of over 1600 English prompts, demonstrate the effectiveness of Parti across a wide variety of categories and difficulty aspects. We also explore and highlight limitations of our models in order to define and exemplify key areas of focus for further improvements. See https://parti.research.google/ for high-resolution images

我们提出了Pathways Autoregressive Text-to-Image (Parti)模型,它能生成高保真的逼真图像,并支持涉及复杂构图和世界知识的内容丰富的合成。Parti将文本到图像的生成视为一个序列到序列的建模问题,类似于机器翻译,将图像标记的序列作为目标输出,而不是另一种语言的文本标记。这种策略可以自然而然地利用先前关于大型语言模型的丰富工作,通过扩大数据和模型的规模,这些模型的能力和性能都有了持续的进步。我们的方法很简单。首先,Parti使用基于Transformer的图像标记器ViT-VQGAN,将图像编码为离散的标记序列。其次,我们通过将编码器-解码器Transformer模型扩展到20B的参数,实现了质量的持续改进,在MS-COCO上,新的最先进的零拍FID得分为7.23,微调的FID得分为3.22。我们对Localized Narratives以及PartiPrompts(P2)的详细分析,这是一个由1600多个英语提示语组成的新的整体基准,证明了Parti在各种类别和难度方面的有效性。我们还探讨并强调了我们的模型的局限性,以确定和示范进一步改进的重点领域。高分辨率图片见 https://parti.research.google/

论文:The ArtBench Dataset: Benchmarking Generative Models with Artworks

论文标题:The ArtBench Dataset: Benchmarking Generative Models with Artworks

论文时间:22 Jun 2022

所属领域:计算机视觉

对应任务:Conditional Image Generation,Image Generation,Unconditional Image Generation,条件性图像生成,图像生成,无条件性图像生成

论文地址:https://arxiv.org/abs/2206.11404

代码实现:https://github.com/liaopeiyuan/artbench

论文作者:Peiyuan Liao, Xiuyu Li, Xihui Liu, Kurt Keutzer

论文简介:We introduce ArtBench-10, the first class-balanced, high-quality, cleanly annotated, and standardized dataset for benchmarking artwork generation./我们推出了ArtBench-10,它是第一个类平衡的、高质量的、干净注释的和标准化的数据集,用于对艺术品生成进行基准测试。

论文摘要:We introduce ArtBench-10, the first class-balanced, high-quality, cleanly annotated, and standardized dataset for benchmarking artwork generation. It comprises 60,000 images of artwork from 10 distinctive artistic styles, with 5,000 training images and 1,000 testing images per style. ArtBench-10 has several advantages over previous artwork datasets. Firstly, it is class-balanced while most previous artwork datasets suffer from the long tail class distributions. Secondly, the images are of high quality with clean annotations. Thirdly, ArtBench-10 is created with standardized data collection, annotation, filtering, and preprocessing procedures. We provide three versions of the dataset with different resolutions (32×32, 256×256, and original image size), formatted in a way that is easy to be incorporated by popular machine learning frameworks. We also conduct extensive benchmarking experiments using representative image synthesis models with ArtBench-10 and present in-depth analysis. The dataset is available at https://github.com/liaopeiyuan/artbench under a Fair Use license.

我们推出ArtBench-10,它是第一个类平衡的、高质量的、干净注释的、标准化的艺术品生成基准数据集。它包括来自10种独特艺术风格的60,000张艺术品图像,每种风格有5,000张训练图像和1,000张测试图像。与以前的艺术品数据集相比,ArtBench-10有几个优点。首先,它是类平衡的,而以前的大多数艺术品数据集都有长尾类分布的问题。第二,图像质量高,注释干净。第三,ArtBench-10是通过标准化的数据收集、注释、过滤和预处理程序创建的。我们提供了三个不同分辨率的数据集版本(32×32、256×256和原始图像大小),其格式化的方式很容易被流行的机器学习框架所采纳。我们还使用ArtBench-10的代表性图像合成模型进行了广泛的基准实验,并进行了深入的分析。该数据集在Fair Use license下可在 https://github.com/liaopeiyuan/artbench 获取使用。

论文:Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models

论文标题:Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models

论文时间:23 Jun 2022

所属领域:计算机视觉

对应任务:Change Detection,Denoising,改变检测,去噪

论文地址:https://arxiv.org/abs/2206.11892

代码实现:https://github.com/wgcban/ddpm-cd

论文作者:Wele Gedara Chaminda Bandara, Nithin Gopalakrishnan Nair, Vishal M. Patel

论文简介:Human civilization has an increasingly powerful influence on the earth system, and earth observations are an invaluable tool for assessing and mitigating the negative impacts./人类文明对地球系统的影响越来越大,而对地观测是评估和减轻负面影响的宝贵工具。

论文摘要:Human civilization has an increasingly powerful influence on the earth system, and earth observations are an invaluable tool for assessing and mitigating the negative impacts. To this end, observing precisely defined changes on Earth’s surface is essential, and we propose an effective way to achieve this goal. Notably, our change detection (CD)/ segmentation method proposes a novel way to incorporate the millions of off-the-shelf, unlabeled, remote sensing images available through different earth observation programs into the training process through denoising diffusion probabilistic models. We first leverage the information from these off-the-shelf, uncurated, and unlabeled remote sensing images by using a pre-trained denoising diffusion probabilistic model and then employ the multi-scale feature representations from the diffusion model decoder to train a lightweight CD classifier to detect precise changes. The experiments performed on four publically available CD datasets show that the proposed approach achieves remarkably better results than the state-of-the-art methods in F1, IoU, and overall accuracy. Code and pre-trained models are available at: https://github.com/wgcban/ddpm-cd

人类文明对地球系统的影响越来越大,而地球观测是评估和减轻负面影响的宝贵工具。为此,观察地球表面精确定义的变化是至关重要的,我们提出了一个有效的方法来实现这一目标。值得注意的是,我们的变化检测(CD)/分割方法提出了一种新的方法,通过去噪扩散概率模型,将数以百万计的现成的、未标记的、可通过不同地球观测项目获得的遥感图像纳入训练过程。我们首先通过使用预先训练好的去噪扩散概率模型来利用这些现成的、未经整理的和未标记的遥感图像的信息,然后采用来自扩散模型解码器的多尺度特征表示来训练一个轻量级的CD分类器来检测精确的变化。在四个公开的CD数据集上进行的实验表明,所提出的方法在F1、IoU和总体准确率方面取得了明显优于最先进方法的结果。代码和预训练的模型可在以下网站获得 https://github.com/wgcban/ddpm-cd

论文:Gender Classification and Bias Mitigation in Facial Images

论文标题:Gender Classification and Bias Mitigation in Facial Images

论文时间:13 Jul 2020

所属领域:计算机视觉

对应任务:Classification,General Classification,分类,通用分类

论文地址:https://arxiv.org/abs/2007.06141

代码实现:https://github.com/Developer-Y/cs-video-courses

论文作者:Wenying Wu, Pavlos Protopapas, Zheng Yang, Panagiotis Michalatos

论文简介:We worked to increase classification accuracy and mitigate algorithmic biases on our baseline model trained on the augmented benchmark database./我们努力提高分类精度,并减轻我们在增强的基准数据库上训练的基线模型的算法偏差。

论文摘要:Gender classification algorithms have important applications in many domains today such as demographic research, law enforcement, as well as human-computer interaction. Recent research showed that algorithms trained on biased benchmark databases could result in algorithmic bias. However, to date, little research has been carried out on gender classification algorithms’ bias towards gender minorities subgroups, such as the LGBTQ and the non-binary population, who have distinct characteristics in gender expression. In this paper, we began by conducting surveys on existing benchmark databases for facial recognition and gender classification tasks. We discovered that the current benchmark databases lack representation of gender minority subgroups. We worked on extending the current binary gender classifier to include a non-binary gender class. We did that by assembling two new facial image databases: 1) a racially balanced inclusive database with a subset of LGBTQ population 2) an inclusive-gender database that consists of people with non-binary gender. We worked to increase classification accuracy and mitigate algorithmic biases on our baseline model trained on the augmented benchmark database. Our ensemble model has achieved an overall accuracy score of 90.39%, which is a 38.72% increase from the baseline binary gender classifier trained on Adience. While this is an initial attempt towards mitigating bias in gender classification, more work is needed in modeling gender as a continuum by assembling more inclusive databases.

性别分类算法在当今许多领域都有重要的应用,如人口统计研究、执法以及人机交互。最近的研究表明,在有偏见的基准数据库上训练的算法可能导致算法的偏见。然而,到目前为止,关于性别分类算法对性别少数群体的偏见的研究还很少,比如LGBTQ和非二元人口,他们在性别表达上有明显的特征。在本文中,我们首先对现有的面部识别和性别分类任务的基准数据库进行了调查。我们发现,目前的基准数据库缺乏对性别少数群体的代表。我们致力于扩展当前的二元性别分类器,以包括非二元性别类。我们通过组装两个新的面部图像数据库来做到这一点。1)一个具有LGBTQ人口子集的种族平衡的包容性数据库 2)一个由非二元性别的人组成的包容性性别数据库。我们努力提高分类精度,并减轻在增强的基准数据库上训练的基线模型的算法偏差。我们的组合模型取得了90.39%的总体准确率,比在Adience上训练的基线二元性别分类器提高了38.72%。虽然这是减轻性别分类偏见的初步尝试,但在通过组建更具包容性的数据库将性别作为一个连续体进行建模方面还需要做更多的工作。

论文:I M Avatar: Implicit Morphable Head Avatars from Videos

论文标题:I M Avatar: Implicit Morphable Head Avatars from Videos

论文时间:CVPR 2022

所属领域:计算机视觉

论文地址:https://arxiv.org/abs/2112.07471

代码实现:https://github.com/zhengyuf/imavatar

论文作者:Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges

论文简介:Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details./传统的三维可变形脸部模型(3DMMs)提供了对表情的细粒度控制,但不能轻易捕捉几何和外观细节。

论文摘要:Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose- related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.

传统的3D可变形脸部模型(3DMMs)提供了对表情的精细控制,但不能轻易捕捉几何和外观细节。神经体积表征接近逼真,但很难制作成动画,也不能很好地概括未见过的表情。为了解决这个问题,我们提出了IMavatar(Implicit Morphable avatar),一种从单眼视频中学习隐性头部头像的新方法。受传统3DMMs提供的细粒度控制机制的启发,我们通过学习的混合形状和皮肤场来表示表情和姿势相关的变形。这些属性是与姿势无关的,并可用于在给定新的表情和姿势参数的情况下对典型的几何和纹理场进行变形。我们采用射线行进法和迭代寻根法来定位每个像素的典型表面交点。我们的一个关键贡献是我们新颖的分析性梯度公式,能够从视频中进行端到端的IMavatars训练。我们从数量和质量上表明,与最先进的方法相比,我们的方法改善了几何学,并覆盖了更完整的表达空间。

论文:A Conversational Paradigm for Program Synthesis

论文标题:A Conversational Paradigm for Program Synthesis

论文时间:25 Mar 2022

所属领域:自然语言处理

对应任务:Language Modelling,Program Synthesis,语言模型,代码生成

论文地址:https://arxiv.org/abs/2203.13474

代码实现:https://github.com/salesforce/CodeGen

论文作者:Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong

论文简介:We train a family of large language models, called CodeGen, on natural language and programming language data./我们在自然语言和编程语言数据上训练一个大型语言模型系列,称为CodeGen。

论文摘要:Program synthesis strives to generate a computer program as a solution to a given problem specification. We propose a conversational program synthesis approach via large language models, which addresses the challenges of searching over a vast program space and user intent specification faced in prior approaches. Our new approach casts the process of writing a specification and program as a multi-turn conversation between a user and a system. It treats program synthesis as a sequence prediction problem, in which the specification is expressed in natural language and the desired program is conditionally sampled. We train a family of large language models, called CodeGen, on natural language and programming language data. With weak supervision in the data and the scaling up of data size and model size, conversational capacities emerge from the simple autoregressive language modeling. To study the model behavior on conversational program synthesis, we develop a multi-turn programming benchmark (MTPB), where solving each problem requires multi-step synthesis via multi-turn conversation between the user and the model. Our findings show the emergence of conversational capabilities and the effectiveness of the proposed conversational program synthesis paradigm. In addition, our model CodeGen (with up to 16B parameters trained on TPU-v4) outperforms OpenAI’s Codex on the HumanEval benchmark. We make the training library JaxFormer including checkpoints available as open source contribution: https://github.com/salesforce/CodeGen

代码生成旨在生成一个计算机程序作为给定问题规格的解决方案。我们提出了一种通过大型语言模型进行对话式程序合成的方法,该方法解决了之前的方法所面临的在巨大的程序空间和用户意图规范上进行搜索的挑战。我们的新方法将编写规范和程序的过程描述为用户和系统之间的多轮对话。它将代码生成视为一个序列预测问题,其中规范是用自然语言表达的,所需的程序是有条件采样的。我们在自然语言和编程语言数据上训练一个大型语言模型系列,称为CodeGen。随着数据中的弱监督以及数据规模和模型规模的扩大,对话能力从简单的自回归语言建模中出现。为了研究对话式程序合成的模型行为,我们开发了一个多轮编程基准(MTPB),其中解决每个问题需要通过用户和模型之间的多轮对话进行多步骤合成。我们的发现显示了对话能力的出现和所提出的对话式程序合成范式的有效性。此外,我们的模型CodeGen(在TPU-v4上训练了多达16B的参数)在HumanEval基准上超过了OpenAI的Codex。我们将包括检查点在内的训练库JaxFormer作为开放源码贡献 https://github.com/salesforce/CodeGen

论文:Elucidating the Design Space of Diffusion-Based Generative Models

论文标题:Elucidating the Design Space of Diffusion-Based Generative Models

论文时间:1 Jun 2022

所属领域:计算机视觉

对应任务:Image Generation,图像生成

论文地址:https://arxiv.org/abs/2206.00364

代码实现:https://github.com/crowsonkb/k-diffusion

论文作者:Tero Karras, Miika Aittala, Timo Aila, Samuli Laine

论文简介:We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices./我们认为,基于扩散的生成模型的理论和实践目前的错综复杂有一些是不必要的,并试图通过提出一个明确区分具体设计选择的设计空间来纠正这种情况。

论文摘要:We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices. This lets us identify several changes to both the sampling and training processes, as well as preconditioning of the score networks. Together, our improvements yield new state-of-the-art FID of 1.79 for CIFAR-10 in a class-conditional setting and 1.97 in an unconditional setting, with much faster sampling (35 network evaluations per image) than prior designs. To further demonstrate their modular nature, we show that our design changes dramatically improve both the efficiency and quality obtainable with pre-trained score networks from previous work, including improving the FID of an existing ImageNet-64 model from 2.07 to near-SOTA 1.55.

我们认为,基于扩散的生成模型的理论和实践目前的错综复杂有一些是不必要的,并试图通过提出一个明确区分具体设计选择的设计空间来纠正这种情况。这让我们确定了采样和训练过程的一些变化,以及得分网络的预处理。我们的改进一起产生了新的最先进的FID,CIFAR-10在类别条件设置中为1.79,在无条件设置中为1.97,采样速度比以前的设计快得多(每幅图像35次网络评估)。为了进一步证明它们的模块化性质,我们表明我们的设计变化极大地提高了以前工作中预训练的评分网络的效率和质量,包括将现有ImageNet-64模型的FID从2.07提高到接近SOTA的1.55。

论文:TAVA: Template-free Animatable Volumetric Actors

论文标题:TAVA: Template-free Animatable Volumetric Actors

论文时间:17 Jun 2022

所属领域:计算机视觉

论文地址:https://arxiv.org/abs/2206.08929

代码实现:https://github.com/facebookresearch/tava

论文作者:RuiLong Li, Julian Tanke, Minh Vo, Michael Zollhofer, Jurgen Gall, Angjoo Kanazawa, Christoph Lassner

论文简介:Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals./由于 TAVA 不需要身体模板,因此它适用于人类以及动物等其他生物。

论文摘要:Coordinate-based volumetric representations have the potential to generate photo-realistic virtual avatars from images. However, virtual avatars also need to be controllable even to a novel pose that may not have been observed. Traditional techniques, such as LBS, provide such a function; yet it usually requires a hand-designed body template, 3D scan data, and limited appearance models. On the other hand, neural representation has been shown to be powerful in representing visual details, but are under explored on deforming dynamic articulated actors. In this paper, we propose TAVA, a method to create T emplate-free Animatable Volumetric Actors, based on neural representations. We rely solely on multi-view data and a tracked skeleton to create a volumetric model of an actor, which can be animated at the test time given novel pose. Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals. Furthermore, TAVA is designed such that it can recover accurate dense correspondences, making it amenable to content-creation and editing tasks. Through extensive experiments, we demonstrate that the proposed method generalizes well to novel poses as well as unseen views and showcase basic editing capabilities.

基于坐标的体积表示法有可能从图像中生成照片般真实的虚拟化身。然而,虚拟头像也需要可控,即使是一个可能没有被观察到的新姿势。传统的技术,如LBS,提供了这样的功能;但它通常需要手工设计的身体模板、3D扫描数据和有限的外观模型。另一方面,神经表征已被证明在代表视觉细节方面非常强大,但在动态铰接演员的变形方面还没有得到充分的探索。在本文中,我们提出了TAVA,一种基于神经表征的、创建无模板的可动画化体积演员的方法。我们仅仅依靠多视图数据和跟踪的骨架来创建演员的体积模型,该模型可以在给定新姿势的测试时间内被动画化。由于TAVA不需要身体模板,它适用于人类和其他生物,如动物。此外,TAVA的设计使其能够恢复准确的密集对应关系,使其适用于内容创建和编辑任务。通过广泛的实验,我们证明了所提出的方法能够很好地概括新的姿势以及未见过的视图,并展示了基本的编辑能力。

我们是 ShowMeAI,致力于传播AI优质内容,分享行业解决方案,用知识加速每一次技术成长!点击查看 历史文章列表,在公众号内订阅话题 #ShowMeAI资讯日报,可接收每日最新推送。点击 专题合辑&电子月刊 快速浏览各专题全集。点击 这里 回复关键字 日报 免费获取AI电子月刊与资料包。

【公式推导】Elucidating the Design Space of Diffusion-Based Generative Models 【论文精读】 (2)引用: Karras T, Aittala M, Aila T, et al. Elucidating the design space of diffusion-based generative models[J]. Advances in Neural Information Processing Systems, 2022, 35: 26565-26577.(3)论文链接: https://doi.org/10.48550/arXiv.2206.00364。 阅读详情

相关推荐

bundled camera paths for videos stabilization 论文翻译 卷帘快门(Rolling Shutter)讲解

作者: http://www.liushuaicheng.org/ MATLAB代码和视频 http://www.liushuaicheng.org/SIGGRAPH2013/index.htm

蜗牛博客 3417

Elucidating the Design Space of Diffusion-Based Generative Models 阅读笔记

文章使用模块化(modular)的思想,分别从采样、训练、score network设计三个方面分析和改进diffusion-based models。

icylling的专栏 5122

【深度学习】k-diffusion,Elucidating the Design Space of Diffusion-Based Generative Models

k-diffusion 提供了 v-diffusion-pytorch、OpenAI diffusion 和 CompVis diffusion 模型的包装器,使它们可以与其采样器和 ODE/SDE 一起使用。k-diffusion 支持 Min-SNR 损失加权的软版本,用于在高分辨率下改进训练,同时所需的超参数比 Karras 等人(2022)使用的损失加权更少。通过这些改进,采样过程不仅在速度上有了显著提升,同时在生成图像的质量上也达到了新的水平。这些改进措施使得扩散模型在实际应用中更加高效和实用。

q742971636的博客 1037

#今日论文推荐#DeepMind谷歌研究员力荐:扩散模型效率&生成质量提升窍门,来自StyleGAN原作者

新晋图像生成王者扩散模型,刚刚诞生没多久。 有关它的理论和实践都还在“野蛮生长”。

w1hsxn的博客 1133

Ubuntu20.04运行Vins-fusion

因已安装ROS noetic,eigen,opencv及ceres等库,这部分环境配置就不细讲,直接下载VINS-FUSION的代码库 下载编译 cd ~/vins-fusion/src git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git cd ../ catkin_make source ~/catkin_ws/devel/setup.bash 编译时会发现一堆错误,别急,无非就是环境冲突问题,一个个解决 1)首先,ROS

xiaojinger_123的博客 1万+

微生物测序分析LEfSe

LEfse分析定义 LEfse分析即LDA Effect Size分析,可以实现多个分组之间的比较,还进行分组比较的内部进行亚组比较分析,从而找到组间在丰度上有显著差异的物种(即biomaker);

宁生信 5万+

【论文阅读】Elucidating the Design Space of Diffusion-Based Generative Models

作者认为,基于扩散的生成模型的理论和实践目前过于复杂,并通过提出一个清晰分离具体设计选择的设计空间来解决这个问题。这使我们能够识别出采样过程和训练过程中的几项改进,以及对分数网络的预处理。我们的改进得到了新的最先进FID(1.79)在CIFAR-10的类条件设置中,以及在无条件设置中的1.97,采样速度比以前的设计快得多(每张图像35次网络评估),将一个预先训练的ImageNet-64模型的FID从2.07提高到接近最先进的1.55,并且在经过改进后重新训练达到了新的最先进水平1.36。

kanon的博客 7181

Intel Realsense windows安装与入门

realsense400系列介绍: https://www.intelrealsense.com/stereo-depth/?utm_source=intelcom_website&utm_medium=button&utm_campaign=day-to-day&utm_content=D400_learn-more_button realsense 435i介绍: ...

hanshuning的专栏 1万+

扩散模型课程第一单元第二部分:扩散模型从零到一

前言于 11 月底正式开课的扩散模型课程正在火热进行中,在中国社区成员们的帮助下,我们组织了「抱抱脸中文本地化志愿者小组」并完成了扩散模型课程的中文翻译,感谢 @darcula1993、@XhrLeokk、@hoi2022、@SuSung-boy 对课程的翻译!如果你还没有开始课程的学习,我们建议你从 第一单元:扩散模型简介 开始。扩散模型从零到一这个 Notebook 我们将展示相同的步骤(向数...

Hugging Face 946

AD24层次原理图设计避坑指南:如何解决端口连接与自动交叉引用问题

本文深入解析AD24层次原理图设计的核心机制,聚焦端口连接错误、自动交叉引用干扰等常见问题。通过对比自上而下与自下而上设计流程,提供端口类型匹配、命名规范等实用解决方案,并指导如何关闭烦人的红色框线视觉干扰,帮助工程师高效避坑,实现复杂系统的模块化设计。

wdx01234567的博客 615

ANSYS材料库实战:手把手教你用APDL命令流定义混凝土和形状记忆合金

本文详细介绍了在ANSYS中使用APDL命令流定义混凝土和形状记忆合金(SMA)的高级建模技巧。通过SOLID65单元和SMA本构模型,工程师可以精确模拟非线性材料行为,包括混凝土的开裂和SMA的超弹性效应。文章提供了完整的命令流示例和求解策略优化建议,帮助用户提升工程仿真效率。

hh234的博客 486

Elucidating the Design Space of Diffusion-Based Generative Models

此篇内容源于论文:Elucidating the Design Space of Diffusion-Based Generative ModelsEDM 构建了一个清晰的扩散模型设计空间,其中各种具体的设计选项解耦开来,能够更精确地分析不同设计选项对于采样、训练各过程的实际影响。

qq_42208244的博客 1254

超声波传感器(CHx01) 学习笔记 Ⅲ - I2C读写操作

CH-x01 通过 2 线 I2C 协议与主机处理器通信

nicole088的专栏 1738

一文读懂扩散模型原理、推断加速和可控生成

来源:智源社区整理:熊宇轩在刚刚过去的2022年,扩散模型(Diffusion Models)成为了深度生成模型中新的SOTA。近期,中国人民大学助理教授李崇轩和清华大学博士生鲍凡在由智源社区主办的“2022大模型创新论坛·峰会-模型技术分论坛”上分享了题为「扩散概率模型:原理、加速推断与可控生成」的报告,介绍了目前大火的扩散模型的基本原理。重点针对扩散模型领域的两大重点问题——「推断加速」、「可...

人工智能学家 6413

基于STM32、HAL库的WF162A 11BA T&R气压传感器 驱动程序设计

/ 转换为7位地址。* @param temperature: 存储读取到的温度值(°C)* @param temperature: 存储读取到的温度值(°C)* @param pressure: 存储读取到的压力值(bar)* @param pressure: 存储读取到的压力值(bar)* @param range: 压力范围(bar)

corlin6688的博客 363

C#工业通信完整项目实战:从协议解析到数据库存储,打造工业级全链路数据闭环(完整可复用源码)

工业自动化的核心诉求是「数据闭环」:从工业设备采集真实生产数据 → 解析校验获得有效数据 → 持久化存储到数据库形成生产台账 → 数据分析/报表展示/异常预警,形成采集→解析→处理→存储→应用的全链路数据流转,这也是工业上位机开发的终极落地形态,区别于单纯的「数据采集」demo。本次实战是一套工业级完整投产级项目,基于C# WinForm + .NET Framework 4.8(工业标配)✅双通信链路完美适配:同时封装TCP网口 + 串口(RS232/485)

这里只有干货:从 Modbus 通信到 MES 对接,从 YOLO 训练到工控机部署,带你搞定工业软件开发全流程。 1037

以蒸馏的名义:“从去噪自编码器到生成模型”重出江湖

©PaperWeekly 原创 ·作者 | 苏剑林单位 | 科学空间研究方向 | NLP、神经网络今天我们分享一下论文《Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation》[1],顾名思义,这是一篇探讨如何更快更好地蒸...

Paper weekly 1269

廖雪峰Python教程[完整版]

廖雪峰的python教程整理的,已经排好版了,而且还是完整的,适合小白

Windows风扇控制终极指南:深度解析FanControl架构设计与性能调优实战

在Windows平台上实现精准风扇控制一直是PC爱好者和专业用户的痛点,传统BIOS控制功能有限且操作繁琐,而第三方软件往往资源占用高且界面复杂。FanControl作为一款免费开源的高度可定制风扇控制软件,通过现代化的架构设计和直观的用户界面,为Windows用户提供了专业级的散热管理解决方案。本指南将从技术架构、部署配置、性能优化到故障排查,全方位解析如何利用FanControl实现PC散热系

gitblog_00022的博客 479

温度PID控制PLC程序

温度PID稳定控制程序,适用于欧姆龙PLC,对其他PLC也具有很大参考价值

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain&#39;s apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath(&#39;DeepLearnToolbox&#39;));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand(&#39;state&#39;,0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W&#39;); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand(&#39;state&#39;,0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = &#39;sigm&#39;;%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, &#39;Too big error&#39;);```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand(&#39;state&#39;,0)sae = saesetup([784 100]);sae.ae{1}.activation_function = &#39;sigm&#39;;sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)&#39;)% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = &#39;sigm&#39;;nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, &#39;Too big error&#39;);```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x&#39;,28,28,60000))/255;test_x = double(reshape(test_x&#39;,28,28,10000))/255;train_y = double(train_y&#39;);test_y = double(test_y&#39;);%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you&#39;ll get around 1.2% errorrand(&#39;state&#39;,0)cnn.layers = { struct(&#39;type&#39;, &#39;i&#39;) %input layer struct(&#39;type&#39;, &#39;c&#39;, &#39;outputmaps&#39;, 6, &#39;kernelsize&#39;, 5) %convolution layer struct(&#39;type&#39;, &#39;s&#39;, &#39;scale&#39;, 2) %sub sampling layer struct(&#39;type&#39;, &#39;c&#39;, &#39;outputmaps&#39;, 12, &#39;kernelsize&#39;, 5) %convolution layer struct(&#39;type&#39;, &#39;s&#39;, &#39;scale&#39;, 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, &#39;Too big error&#39;);```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand(&#39;state&#39;,0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, &#39;Too big error&#39;);%% ex2 neural net with L2 weight decayrand(&#39;state&#39;,0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, &#39;Too big error&#39;);%% ex3 neural net with dropoutrand(&#39;state&#39;,0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, &#39;Too big error&#39;);%% ex4 neural net with sigmoid activation functionrand(&#39;state&#39;,0)nn = nnsetup([784 100 10]);nn.activation_function = &#39;sigm&#39;; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, &#39;Too big error&#39;);%% ex5 plotting functionalityrand(&#39;state&#39;,0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = &#39;softmax&#39;; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, &#39;Too big error&#39;);%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand(&#39;state&#39;,0)nn = nnsetup([784 20 10]); nn.output = &#39;softmax&#39;; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, &#39;Too big error&#39;);```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

[论文笔记]Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models

[论文笔记]Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models

weixin_42656681的博客 1652

Camera2Demo

在Camera2 的基础上进行封装,使得Camera2一堆初始化逻辑与Activity分离,降低耦合度,完成预览拍照并保存的操作

2022年 change detection遥感图像变化检测 论文附代码

动机:可用于训练CD模型的标注训练图像较少,应该注重从数百万免费可用的、无标记的、无策展的遥感图像中挖掘尽可能多的信息,以提高CD的准确性和鲁棒性。动机:可用于训练CD模型的标注训练图像较少,应该注重从数百万免费可用的、无标记的、无策展的遥感图像中挖掘尽可能多的信息,以提高CD的准确性和鲁棒性。介绍:提出了一种新的方法,通过去噪扩散概率模型,将通过不同的地球观测计划获得的数百万现成的、未标记的遥感图像合并到训练过程中。1)随着遥感图像分辨率的提高,高分辨率图像中包含的丰富语义信息没有得到充分利用。

qq_40994007的博客 1万+

CS231n.github.io前沿综述:计算机视觉2025研究热点终极指南

CS231n.github.io作为斯坦福大学著名的计算机视觉课程公开资料库,持续跟踪和呈现计算机视觉领域的最新发展。在2025年,计算机视觉研究呈现出几个明显的热点趋势,包括Transformer架构的全面渗透、自监督学习的规模化应用、扩散模型的突破性进展,以及神经辐射场(NeRF)技术的成熟化。这些前沿技术正在重新定义计算机视觉的边界,为人工智能的实际应用开辟了新的可能性。 ## 🤖 Tr

gitblog_00151的博客 850

如何快速构建3D数字人:IMavatar从视频生成隐式可变形头像的完整指南

IMavatar是一个强大的开源项目,专门用于从视频中生成隐式可变形头部头像(Implicit Morphable Head Avatars)。这个CVPR 2022的口头论文项目让你能够从简单的视频输入创建高质量的3D数字人模型,无需复杂的3D扫描设备。无论你是虚拟形象开发者、数字内容创作者还是AI研究者,这个终极教程将带你快速上手IMavatar,掌握从安装到生成3D头像的全流程。 ## ?

gitblog_00187的博客 607

探索未来驾驶:Nocturne——一款面向真实世界的多智能体学习驾驶模拟器

探索未来驾驶:Nocturne——一款面向真实世界的多智能体学习驾驶模拟器 nocturneA data-driven, fast driving simulator for multi-agent coordination under partial observability. 项目地址:https://gitcode.com/gh_mirrors/no/nocturne 项目介绍 在研究自...

gitblog_00002的博客 510

从2D图像生成3D模型,基于Im2Avatar

原文链接:Im2Avatar: Colorful 3D Reconstruction from a Single Imagegithub.com这个模型不仅可以生成3D模型,还可以生成颜色,现在我就介绍一下生成形状的部分,这个网络不是end to end的形式所以形状和颜色是分别训练的。源代码其实是python2写的,稍微改一下就可以用在python3中。它的网络我大致列一下,也欢迎指正:编码:解...

凌空的桨 1万+

从视频到3D头像:IMavatar数据预处理全流程(附DECA与面部解析工具)

IMavatar是CVPR 2022的开源项目,能够从视频中生成高逼真度的隐式可变形头部头像。本文将详细介绍如何使用IMavatar的预处理工具链,将普通视频转化为可用于3D头像建模的结构化数据,包括视频裁剪、背景分割、面部特征提取等关键步骤。 ## 准备工作:环境配置与项目结构 在开始预处理前,请确保已克隆项目仓库并安装依赖: ```bash git clone https://gitcod

gitblog_00616的博客 121

从理论到实践:GigaChat3.1-10B-A1.8B-GGUF混合专家架构深度解析

GigaChat3.1-10B-A1.8B-GGUF是GigaChat 3.1系列的轻量级指令模型,基于混合专家(Mixture-of-Experts, MoE)架构构建,总参数达100亿,活跃参数18亿,专为快速多语言助手任务、推理、代码生成、工具调用和产品级部署优化。本文将从理论到实践,全面解析这一创新架构的核心优势与应用方法。 ## 什么是混合专家(MoE)架构?🤔 混合专家架构是一种

gitblog_00915的博客 1024
上一篇: 人工智能 | ShowMeAI资讯日报 #2022.06.28
下一篇: 人工智能 | ShowMeAI资讯日报 #2022.06.30
ShowMeAI
博客等级 码龄5年 6514粉丝 672原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

ShowMeAI

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值