(venv) D:\Audio2Face\Audio2Face-3D-SDK>trtexec --version
&&&& RUNN
ING TensorRT.trtexec [TensorRT v101401] [b48] # trtexec --version
[11/27/2025-14:33:27] [I] TF32 is enabled by default. Add --
noTF32 flag to fur
ther improve accuracy
with some performance cost.
=== Model Options ===
--onnx=<file> ONNX model
=== Build Options ===
--m
inShapes=spec Build
with dynamic shapes us
ing a profile
with the m
in shapes prov
ided
--optShapes=spec Build
with dynamic shapes us
ing a profile
with the opt shapes prov
ided
--maxShapes=spec Build
with dynamic shapes us
ing a profile
with the max shapes prov
ided
--m
inShapesCalib=spec Calibrate
with dynamic shapes us
ing a profile
with the m
in shapes prov
ided
--optShapesCalib=spec Calibrate
with dynamic shapes us
ing a profile
with the opt shapes prov
ided
--maxShapesCalib=spec Calibrate
with dynamic shapes us
ing a profile
with the max shapes prov
ided
Note: All three of m
in, opt and max shapes must be supplied.
However, if only opt shapes is supplied
then it will be expanded so
that m
in shapes and max shapes are set to
the same values as opt shapes.
Input names can be wrapped
with escaped s
ingle quotes (ex: '
Input:0').
Example
input shapes spec:
input0:1x3x256x256,
input1:1x3x128x128
For scalars (0-D shapes), use
input0:scalar or simply
input0:
with noth
ing after
the colon.
Each
input shape is supplied as a key-value pair where key is
the input name and
value is
the dimensions (
includ
ing
the batch dimension) to be used for that
input.
Each key-value pair has
the key and value separated us
ing a colon (:).
Multiple
input shapes can be prov
ided via comma-separated key-value pairs, and each
input name can
conta
in at most one wildcard ('*') character.
--
inputIOFormats=spec Type and format of each of
the input tensors (default = all
inputs
in fp32:chw)
See --outputIOFormats help for
the grammar of type and format list.
Note: If this option is specified, please set comma-separated types and formats for all
inputs follow
ing
the same order as network
inputs
ID (even if only one
input
needs specify
ing IO format) or set
the type and format once for broadcast
ing.
--outputIOFormats=spec Type and format of each of
the output tensors (default = all outputs
in fp32:chw)
Note: If this option is specified, please set comma-separated types and formats for all
outputs follow
ing
the same order as network outputs
ID (even if only one output
needs specify
ing IO format) or set
the type and format once for broadcast
ing.
IO Formats: spec ::= IOfmt[","spec]
IOfmt ::= type:fmt
type ::= "fp32"|"fp16"|"bf16"|"
int32"|"
int
64"|"
int8"|"u
int8"|"bool"
fmt ::= ("chw"|"chw2"|"hwc8"|"chw4"|"chw16"|"chw32"|"dhwc8"|
"cdhw32"|"hwc"|"dla_l
inear"|"dla_hwc4"|"hwc16"|"dhwc")["+"fmt]
--memPoolSize=poolspec Specify
the size constra
ints of
the designated memory pool(s)
Supports
the follow
ing base-2 suffixes: B (Bytes), G (Gibibytes), K (Kibibytes), M (Mebibytes).
If
none of suffixes is appended,
the defualt unit is
in MiB.
Note: Also accepts decimal sizes, e.g. 0.25M. Will be rounded down to
the nearest
integer bytes.
In particular, for dlaSRAM
the bytes will be rounded down to
the nearest power of 2.
Pool constra
int: poolspec ::= poolfmt[","poolspec]
poolfmt ::= pool:size
pool ::= "workspace"|"dlaSRAM"|"dlaLocalDRAM"|"dlaGlobalDRAM"|"tacticSharedMem"
--profil
ingVerbosity=mode Specify profil
ing verbosity. mode ::= layer_names_only|detailed|
none (default = layer_names_only).
Please only assign once.
--avgTim
ing=M Set
the number of times averaged
in each iteration for kernel selection (default = 8)
--refit Mark
the eng
ine as refittable. This will allow
the inspection of refittable layers
and weights
within the eng
ine.
--stripWeights Strip weights from plan. This flag works
with ei
ther refit or refit
with identical weights. Default
to latter, but you can switch to
the former by enabl
ing both --stripWeights and --refit at
the same
time.
--stripAllWeights Alias for comb
ining
the --refit and --stripWeights options. It marks all weights as refittable,
disregard
ing any performance impact. Additionally, it strips all refittable weights after
the
eng
ine is built.
--weightless [Deprecated] this k
nob has been deprecated. Please use --stripWeights
--versionCompatible, --vc Mark
the eng
ine as version compatible. This allows
the eng
ine to be used
with newer versions
of TensorRT on
the same host OS, as well as TensorRT's dispatch and lean runtimes.
--plug
inInstance
Norm, --pi Set `kNATIVE_
INSTANCE
NORM` to false
in the ONNX parser. This will cause
the ONNX parser to use
a plug
in Instance
Norm implementation over
the native implementation when pars
ing.
--u
int8AsymmetricQuantizationDLA Set `kENABLE_U
INT8_AND_ASYMMETRIC_QUANTIZATION_DLA` to true
in the ONNX parser. This directs
the
onnx parser to allow U
INT8 as a quantization data type and import zero po
int values directly
without convert
ing to float type or all-zero values. Should only be set
with DLA software version
>= 3.16.
--useRuntime=runtime TensorRT runtime to execute eng
ine. "lean" and "dispatch" require load
ing VC eng
ine and do
not support build
ing an eng
ine.
runtime::= "full"|"lean"|"dispatch"
--leanDLLPath=<file> External lean runtime DLL to use
in version compatible mode.
--excludeLeanRuntime When --versionCompatible is enabled, this flag
indicates that
the generated eng
ine should
not
include an embedded lean runtime. If this is set,
the user must explicitly specify a
val
id lean runtime to use when load
ing
the eng
ine.
--monitorMemory Enable memory monitor report for debugg
ing usage. (default = disabled)
--sparsity=spec Control sparsity (default = disabled).
Sparsity: spec ::= "disable", "enable", "force"
Note: Description about each of
these options is as below
disable = do
not enable sparse tactics
in the builder (this is
the default)
enable = enable sparse tactics
in the builder (but
these tactics will only be
cons
idered if
the weights have
the right sparsity pattern)
force = enable sparse tactics
in the builder and force-overwrite
the weights to have
a sparsity pattern (even if you loaded a model yourself)
[Deprecated] this k
nob has been deprecated.
Please use <polygraphy surgeon prune> to rewrite
the weights.
--
noTF32 Disable tf32 precision (default is to enable tf32,
in addition to fp32)
--fp16 Enable fp16 precision,
in addition to fp32 (default = disabled)
--bf16 Enable bf16 precision,
in addition to fp32 (default = disabled)
--
int8 Enable
int8 precision,
in addition to fp32 (default = disabled)
--fp8 Enable fp8 precision,
in addition to fp32 (default = disabled)
--
int4 Enable
int4 precision,
in addition to fp32 (default = disabled)
--best Enable all precisions to achieve
the best performance (default = disabled)
Note: --fp16, --bf16, --
int8, --fp8, --
int4, --best are deprecated and superseded by strong typ
ing.
The AutoCast tool (https://nv
idia.github.io/TensorRT-Model-Optimizer/gu
ides/8_autocast.html)
can be used to convert
the network to be strongly typed.
--stronglyTyped Create a strongly typed network. (default = disabled)
--directIO [Deprecated] Avo
id reformatt
ing at network boundaries. (default = disabled)
--precisionConstra
ints=spec Control precision constra
int sett
ing. (default =
none)
Precision Constra
ints: spec ::= "
none" | "obey" | "prefer"
none =
no constra
ints
prefer = meet precision constra
ints set by --layerPrecisions/--layerOutputTypes if possible
obey = meet precision constra
ints set by --layerPrecisions/--layerOutputTypes or fail
o
therwise
--layerPrecisions=spec Control per-layer precision constra
ints. Effective only when precisionConstra
ints is set to
"obey" or "prefer". (default =
none)
The specs are read left-to-right, and later ones overr
ide earlier ones. Each layer name can
conta
in at most one wildcard ('*') character.
Per-layer precision spec ::= layerPrecision[","spec]
layerPrecision ::= layerName":"precision
precision ::= "fp32"|"fp16"|"bf16"|"
int32"|"
int8"
--layerOutputTypes=spec Control per-layer output type constra
ints. Effective only when precisionConstra
ints is set to
"obey" or "prefer". (default =
none
The specs are read left-to-right, and later ones overr
ide earlier ones. Each layer name can
conta
in at most one wildcard ('*') character. If a layer has more than
one output,
then multiple types separated by "+" can be prov
ided for this layer.
Per-layer output type spec ::= layerOutputTypes[","spec]
layerOutputTypes ::= layerName":"type
type ::= "fp32"|"fp16"|"bf16"|"
int32"|"
int8"["+"type]
--layerDeviceTypes=spec Specify layer-specific device type.
The specs are read left-to-right, and later ones overr
ide earlier ones. If a layer does
not have
a device type specified,
the layer will opt for
the default device type.
Per-layer device type spec ::= layerDeviceTypePair[","spec]
layerDeviceTypePair ::= layerName":"deviceType
deviceType ::= "GPU"|"DLA"
--decomposableAttentions=spec Specify decomposable attentions by comma-separated names.
The specs are read left-to-right, and later ones overr
ide earlier ones. Each layer name can
conta
in at most one wildcard ('*') character.
--calib=<file> Read
INT8 calibration cache file
--safe Enable build safety certified eng
ine, --stronglyTyped will be enabled by default
with this option.
If DLA is enabled, --buildDLAStandalone will be specified
--dumpKernelText Dump
the kernel text to a file, only available when --safe is enabled
--buildDLAStandalone Enable build DLA standalone loadable which can be loaded by cuDLA, when this option is enabled,
--allowGPUFallback is disallowed and --skip
Inference is enabled by default. Additionally,
specify
ing --
inputIOFormats and --outputIOFormats restricts I/O data type and memory layout
(default = disabled)
--allowGPUFallback When DLA is enabled, allow GPU fallback for unsupported layers (default = disabled)
--consistency Perform consistency check
ing on safety certified eng
ine
--restricted Enable safety scope check
ing
with kSAFETY_SCOPE build flag
--saveEng
ine=<file> Save
the serialized eng
ine
--loadEng
ine=<file> Load a serialized eng
ine
--asyncFileReader Load a serialized eng
ine us
ing async stream reader. Should be comb
ined
with --loadEng
ine.
--getPlanVersio
nOnly Pr
int TensorRT version when loaded plan was created. Works
without deserialization of
the plan.
Use toge
ther
with --loadEng
ine. Supported only for eng
ines created
with 8.6 and forward.
--tacticSources=tactics Specify
the tactics to be used by add
ing (+) or remov
ing (-) tactics from
the default
tactic sources (default = all available tactics).
Note: Currently only cuDNN, cuBLAS, cuBLAS-LT, and edge mask convolutions are listed as optional
tactics.
Tactic Sources: tactics ::= tactic[","tactics]
tactic ::= (+|-)lib
lib ::= "CUBLAS"|"CUBLAS_LT"|"CUDNN"|"EDGE_MASK_CONVOLUTIONS"
|"JIT_CONVOLUTIONS"
For example, to disable cudnn and enable cublas: --tacticSources=-CUDNN,+CUBLAS
--
noBuilderCache Disable tim
ing cache
in builder (default is to enable tim
ing cache)
--
noCompilationCache Disable Compilation cache
in builder, and
the cache is part of tim
ing cache (default is to enable compilation cache)
--errorOnTim
ingCacheMiss Emit error when a tactic be
ing timed is
not present
in the tim
ing cache (default = false)
--tim
ingCacheFile=<file> Save/load
the serialized global tim
ing cache
--preview=features Specify preview feature to be used by add
ing (+) or remov
ing (-) preview features from
the default
Preview Features: features ::= feature[","features]
feature ::= (+|-)flag
flag ::= "aliasedPlug
inIO1003"
|"runtimeActivationResize"
|"profileShar
ing0806"
--builderOptimizationLevel Set
the builder optimization level. (default is 3)
A Higher level allows TensorRT to spend more time search
ing for better optimization strategy.
Val
id values
include
integers from 0 to
the maximum optimization level, which is currently 5.
--maxTactics Set
the maximum number of tactics to time when
there is a choice of tactics. (default is -1)
Larger number of tactics allow TensorRT to spend more build
ing time on evaluat
ing tactics.
Default value -1 means TensorRT can dec
ide
the number of tactics based on its own heuristic.
--hardwareCompatibilityLevel=mode Make
the eng
ine file compatible
with o
ther GPU architectures. (default =
none)
Hardware Compatibility Level: mode ::= "
none" | "ampere+" | "sameComputeCapability"
none =
no compatibility
ampere+ = compatible
with Ampere and newer GPUs
sameComputeCapability = compatible
with GPUs that have
the same Compute Capability version
--runtimePlatform=platform Set
the target platform for runtime execution. (default = SameAsBuild)
When this option is enabled, --skip
Inference is enabled by default.
RuntimePlatfrom: platform ::= "SameAsBuild" | "W
indowsAMD
64"
SameAsBuild =
no requirement for cross-platform compatibility.
W
indowsAMD
64 = set
the target platform for eng
ine execution as W
indows AMD
64 system
--tempdir=<dir> Overr
ides
the default temporary directory TensorRT will use when creat
ing temporary files.
See IRuntime::setTemporaryDirectory API documentation for more
information.
--tempfileControls=controls Controls what TensorRT is allowed to use when creat
ing temporary executable files.
Should be a comma-separated list
with entries
in the format (
in_memory|temporary):(allow|deny).
in_memory: Controls whe
ther TensorRT is allowed to create temporary
in-memory executable files.
temporary: Controls whe
ther TensorRT is allowed to create temporary executable files
in the
file
system (
in the directory given by --tempdir).
For example, to allow
in-memory files and disallow temporary files:
--tempfileControls=
in_memory:allow,temporary:deny
If a flag is unspecified,
the default behavior is "allow".
--maxAuxStreams=N Set maximum number of auxiliary streams per
inference stream that TRT is allowed to use to run
kernels
in parallel if
the network conta
ins ops that can run
in parallel,
with the cost of more
memory usage. Set this to 0 for optimal memory usage. (default = us
ing heuristics)
--profile Build
with dynamic shapes us
ing a profile
with the m
in/max/opt shapes prov
ided. Can be specified
multiple times to create multiple profiles
with contiguous
index.
(ex: --profile=0 --m
inShapes=<spec> --optShapes=<spec> --maxShapes=<spec> --profile=1 ...)
--calibProfile Select
the optimization profile to calibrate by
index. (default = 0)
--allowWeightStream
ing Enable a weight stream
ing eng
ine. Must be specified
with --stronglyTyped. TensorRT will disable
weight stream
ing at runtime unless --weightStream
ingBudget is specified.
--markDebug Specify list of names of tensors to be marked as debug tensors. Separate names
with a comma
--markUnfusedTensorsAsDebugTensors Mark unfused tensors as debug tensors
--til
ingOptimizationLevel Set
the til
ing optimization level. (default is 0)
A Higher level allows TensorRT to spend more time search
ing for better optimization strategy.
Val
id values
include
integers from 0 to
the maximum til
ing optimization level(3).
--l2LimitForTil
ing Set
the L2 cache usage limit for til
ing optimization(default is -1)
--remoteAutoTun
ingConfig Set
the remote auto tun
ing config. Must be specified
with --safe.
Format: protocol://username[:password]@hostname[:port]?param1=value1¶m2=value2
Example: ssh://user:pass@192.0.2.100:22?remote_exec_path=/opt/tensorrt/b
in&remote_lib_path=/opt/tensorrt/lib
--refitFromOnnx Refit
the loaded eng
ine
with the weights from
the prov
ided ONNX model.
The model should be
identical to
the one used to generate
the eng
ine.
===
Inference Options ===
--shapes=spec Set
input shapes for dynamic shapes
inference
inputs.
Note:
Input names can be wrapped
with escaped s
ingle quotes (ex: '
Input:0').
Example
input shapes spec:
input0:1x3x256x256,
input1:1x3x128x128
For scalars (0-D shapes), use
input0:scalar or simply
input0:
with noth
ing after
the colon.
Each
input shape is supplied as a key-value pair where key is
the input name and
value is
the dimensions (
includ
ing
the batch dimension) to be used for that
input.
Each key-value pair has
the key and value separated us
ing a colon (:).
Multiple
input shapes can be prov
ided via comma-separated key-value pairs, and each
input
name can conta
in at most one wildcard ('*') character.
--load
Inputs=spec Load
input values from files (default = generate random
inputs).
Input names can be wrapped
with s
ingle quotes (ex: '
Input:0')
Input values spec ::= Ival[","spec]
Ival ::= name":"file
Consult
the README for more
information on generat
ing files for custom
inputs.
--iterations=N Run at least N
inference iterations (default = 10)
--warmUp=N Run for N milliseconds to warmup before measur
ing performance (default = 200)
--duration=N Run performance measurements for at least N seconds wallclock time (default = 3)
If -1 is specified,
inference will keep runn
ing unless stopped manually
--sleepTime=N Delay
inference start
with a gap of N milliseconds between launch and compute (default = 0)
--
idleTime=N Sleep N milliseconds between two cont
inuous iterations(default = 0)
--
infStreams=N
Instantiate N execution contexts to run
inference concurrently (default = 1)
--exposeDMA Serialize DMA transfers to and from device (default = disabled).
--
noDataTransfers Disable DMA transfers to and from device (default = enabled).
Note some device-to-host
data transfers will rema
in if output dump
ing is enabled via
the --dumpOutput or
--exportOutput flags.
--useManagedMemory Use managed memory
instead of separate host and device allocations (default = disabled).
--useSp
inWait Actively synchronize on GPU events. This option may decrease synchronization time but
increase CPU usage and power (default = disabled)
--threads Enable multithread
ing to drive eng
ines
with independent threads or speed up refitt
ing (default = disabled)
--useCudaGraph Use CUDA graph to capture eng
ine execution and
then launch
inference (default = disabled).
This flag may be ig
nored if
the graph capture fails.
--timeDeserialize Time
the amount of time it takes to deserialize
the network and exit.
--timeRefit Time
the amount of time it takes to refit
the eng
ine before
inference.
--separateProfileRun Do
not attach
the profiler
in the benchmark run; if profil
ing is enabled, a second profile run will be executed (default = disabled)
--skip
Inference Exit after
the eng
ine has been built and skip
inference perf measurement (default = disabled)
--persistentCacheRatio Set
the persistentCacheLimit
in ratio, 0.5 represent half of max persistent L2 size (default = 0)
--useProfile Set
the optimization profile for
the inference context (default = 0 ).
--allocationStrategy=spec Specify how
the internal device memory for
inference is allocated.
Strategy: spec ::= "static"|"profile"|"runtime"
static = Allocate device memory based on max size across all profiles.
profile = Allocate device memory based on max size of
the current profile.
runtime = Allocate device memory based on
the actual
input shapes.
--saveDebugTensors Specify list of names of tensors to turn on
the debug state
and filename to save raw outputs to.
These tensors must be specified as debug tensors dur
ing build time.
Input values spec ::= Ival[","spec]
Ival ::= name":"file
--saveAllDebugTensors Save all debug tensors to files.
Includ
ing debug tensors marked by --markDebug and --markUnfusedTensorsAsDebugTensors
Multiple file formats can be saved simultaneously.
Input values spec ::= format[","format]
format ::= "summary"|"numpy"|"str
ing"|"raw"
--weightStream
ingBudget Set
the maximum amount of GPU memory TensorRT is allowed to use for weights.
It can take on
the follow
ing values:
-2: (default) Disable weight stream
ing at runtime.
-1: TensorRT will automatically dec
ide
the budget.
0-100%: Percentage of streamable weights that res
ide on
the GPU.
0% saves
the most memory but will have
the worst performance.
Requires
the '%' character.
>=0B:
The exact amount of streamable weights that res
ide on
the GPU. Supports
the
follow
ing base-2 suffixes: B (Bytes), G (Gibibytes), K (Kibibytes), M (Mebibytes).
=== Report
ing Options ===
--verbose Use verbose logg
ing (default = false)
--avgRuns=N Report performance measurements averaged over N consecutive iterations (default = 10)
--percentile=P1,P2,P3,... Report performance for
the P1,P2,P3,... percentages (0<=P_i<=100, 0 represent
ing max perf, and 100 represent
ing m
in perf; (default = 90,95,99%)
--dumpRefit Pr
int
the refittable layers and weights from a refittable eng
ine
--dumpOutput Pr
int
the output tensor(s) of
the last
inference iteration (default = disabled)
--dumpRawB
ind
ingsToFile Pr
int
the input/output tensor(s) of
the last
inference iteration to file(default = disabled)
--dumpProfile Pr
int profile
information per layer (default = disabled)
--dumpLayer
Info Pr
int layer
information of
the eng
ine to console (default = disabled)
--dumpOptimizationProfile Pr
int
the optimization profile(s)
information (default = disabled)
--exportTimes=<file> Write
the tim
ing results
in a json file (default = disabled)
--exportOutput=<file> Write
the output tensors to a json file (default = disabled)
--exportProfile=<file> Write
the profile
information per layer
in a json file (default = disabled)
--exportLayer
Info=<file> Write
the layer
information of
the eng
ine
in a json file (default = disabled)
===
System Options ===
--device=N Select cuda device N (default = 0)
--useDLACore=N Select DLA core N for layers that support DLA (default =
none)
--staticPlug
ins Plug
in library (.so) to load statically (can be specified multiple times)
--dynamicPlug
ins Plug
in library (.so) to load dynamically and may be serialized
with the eng
ine if
they are
included
in --setPlug
insToSerialize (can be specified multiple times)
--setPlug
insToSerialize Plug
in library (.so) to be serialized
with the eng
ine (can be specified multiple times)
--ig
noreParsedPlug
inLibs By default, when build
ing a version-compatible eng
ine, plug
in libraries specified by
the ONNX parser
are implicitly serialized
with the eng
ine (unless --excludeLeanRuntime is specified) and loaded dynamically.
Enable this flag to ig
nore
these plug
in libraries
instead.
--safetyPlug
ins Plug
in library (.so) for TensorRT auto safety to manually load safety plug
ins specified by
the command l
ine arguments.
Example: --safetyPlug
ins=/path/to/plug
in_lib.so[plug
inNamespace1::plug
in1,plug
inNamespace2::plug
in2].
The option can be specified multiple times
with different plug
in libraries.
=== Help ===
--help, -h Pr
int this message
[11/27/2025-14:33:27] [E] Model miss
ing or format
not recognized
&&&& FAILED TensorRT.trtexec [TensorRT v101401] [b48] # trtexec --version
当在进行SDK下载时遇到中途出错的问题时，本文提供了一个有效的解决策略：清理运行配置中可能存在的错误设置，特别是对于缺少Debug选项的情况。通过此方法，可以避免下载过程中的常见错误，确保SDK安装的顺利进行。
扫一扫
1573
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
