(venv) D:\Audio2Face\Audio2Face
-3D
-SDK>trtexec
--version
&&&& RUNNING TensorRT.trtexec [TensorRT v101401] [b48] # trtexec
--version
[11/27/2025
-14:33:27] [I] TF32
is enabled by default. Add
--noTF32 flag to further improve accuracy with some per
formance cost.
=== Model Options ===
--onnx=<file> ONNX model
=== Build Options ===
--minShapes=spec Build with dynamic shapes using a profile with the min shapes provided
--optShapes=spec Build with dynamic shapes using a profile with the opt shapes provided
--maxShapes=spec Build with dynamic shapes using a profile with the max shapes provided
--minShapesCalib=spec Calibrate with dynamic shapes using a profile with the min shapes provided
--optShapesCalib=spec Calibrate with dynamic shapes using a profile with the opt shapes provided
--maxShapesCalib=spec Calibrate with dynamic shapes using a profile with the max shapes provided
Note: All three of min, opt and max shapes must be supplied.
However, if only opt shapes
is supplied then it will be expanded so
that min shapes and max shapes are set to the same values as opt shapes.
Input names can be wrapped with escaped single quotes (ex: 'Input:0').
Example input shapes spec: input0:1x3x256x256,input1:1x3x128x128
For scalars (0
-D shapes), use input0:scalar or simply input0: with
nothing after the colon.
Each input shape
is supplied as a key
-value pair where key
is the input name and
value
is the dimensions (including the batch dimension) to be used
for that input.
Each key
-value pair has the key and value separated using a colon (:).
Multiple input shapes can be provided via comma
-separated key
-value pairs, and each input name can
contain at most one wildcard ('*') character.
--inputIO
Formats=spec Type and
format of each of the input tensors (default = all inputs in fp32:chw)
See
--outputIO
Formats help
for the grammar of type and
format l
ist.
Note: If th
is option
is specified, please set comma
-separated types and
formats
for all
inputs following the same order as network inputs ID (even if only one input
needs specifying IO
format) or set the type and
format once
for broadcasting.
--outputIO
Formats=spec Type and
format of each of the output tensors (default = all outputs in fp32:chw)
Note: If th
is option
is specified, please set comma
-separated types and
formats
for all
outputs following the same order as network outputs ID (even if only one output
needs specifying IO
format) or set the type and
format once
for broadcasting.
IO
Formats: spec ::= IOfmt[","spec]
IOfmt ::= type:fmt
type ::= "fp32"|"fp16"|"bf16"|"int32"|"int64"|"int8"|"uint8"|"bool"
fmt ::= ("chw"|"chw2"|"hwc8"|"chw4"|"chw16"|"chw32"|"dhwc8"|
"cdhw32"|"hwc"|"dla_linear"|"dla_hwc4"|"hwc16"|"dhwc")["+"fmt]
--memPoolSize=poolspec Specify the size constraints of the designated memory pool(s)
Supports the following base
-2 suffixes: B (Bytes), G (Gibibytes), K (Kibibytes), M (Mebibytes).
If none of suffixes
is appended, the defualt unit
is in MiB.
Note: Also accepts decimal sizes, e.g. 0.25M. Will be rounded down to the nearest integer bytes.
In particular,
for dlaSRAM the bytes will be rounded down to the nearest power of 2.
Pool constraint: poolspec ::= poolfmt[","poolspec]
poolfmt ::= pool:size
pool ::= "workspace"|"dlaSRAM"|"dlaLocalDRAM"|"dlaGlobalDRAM"|"tacticSharedMem"
--profilingVerbosity=mode Specify profiling verbosity. mode ::= layer_names_only|detailed|none (default = layer_names_only).
Please only assign once.
--avgTiming=M Set the number of times averaged in each iteration
for kernel selection (default = 8)
--refit Mark the engine as refittable. Th
is will allow the inspection of refittable layers
and weights within the engine.
--stripWeights Strip weights from plan. Th
is flag works with either refit or refit with identical weights. Default
to latter, but you can switch to the
former by enabling both
--stripWeights and
--refit at the same
time.
--stripAllWeights Alias
for combining the
--refit and
--stripWeights options. It marks all weights as refittable,
d
isregarding any per
formance impact. Additionally, it strips all refittable weights after the
engine
is built.
--weightless [Deprecated] th
is knob has been deprecated. Please use
--stripWeights
--versionCompatible,
--vc Mark the engine as version compatible. Th
is allows the engine to be used with newer versions
of TensorRT on the same host OS, as well as TensorRT's d
ispatch and lean runtimes.
--pluginInstanceNorm,
--pi Set `kNATIVE_INSTANCENORM` to false in the ONNX parser. Th
is will cause the ONNX parser to use
a plugin InstanceNorm implementation over the native implementation when parsing.
--uint8AsymmetricQuantizationDLA Set `k
ENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA` to true in the ONNX parser. Th
is directs the
onnx parser to allow UINT8 as a quantization data type and import zero point values directly
without converting to float type or all
-zero values. Should only be set with DLA software version
>= 3.16.
--useRuntime=runtime TensorRT runtime to execute engine. "lean" and "d
ispatch" require loading VC engine and do
not support building an engine.
runtime::= "full"|"lean"|"d
ispatch"
--leanDLLPath=<file> External lean runtime DLL to use in version compatible mode.
--excludeLeanRuntime When
--versionCompatible
is enabled, th
is flag indicates that the generated engine should
not include an embedded lean runtime. If th
is is set, the user must explicitly specify a
valid lean runtime to use when loading the engine.
--monitorMemory
Enable memory monitor report
for debugging
usage. (default = d
isabled)
--sparsity=spec Control sparsity (default = d
isabled).
Sparsity: spec ::= "d
isable", "
enable", "
force"
Note: Description about each of these options
is as below
d
isable = do
not enable sparse tactics in the builder (th
is is the default)
enable =
enable sparse tactics in the builder (but these tactics will only be
considered if the weights have the right sparsity pattern)
force =
enable sparse tactics in the builder and
force
-overwrite the weights to have
a sparsity pattern (even if you loaded a model yourself)
[Deprecated] th
is knob has been deprecated.
Please use <polygraphy surgeon prune> to rewrite the weights.
--noTF32 D
isable tf32 prec
ision (default
is to
enable tf32, in addition to fp32)
--fp16
Enable fp16 prec
ision, in addition to fp32 (default = d
isabled)
--bf16
Enable bf16 prec
ision, in addition to fp32 (default = d
isabled)
--int8
Enable int8 prec
ision, in addition to fp32 (default = d
isabled)
--fp8
Enable fp8 prec
ision, in addition to fp32 (default = d
isabled)
--int4
Enable int4 prec
ision, in addition to fp32 (default = d
isabled)
--best
Enable all prec
isions to achieve the best per
formance (default = d
isabled)
Note:
--fp16,
--bf16,
--int8,
--fp8,
--int4,
--best are deprecated and superseded by strong typing.
The AutoCast tool (https://nvidia.github.io/TensorRT
-Model
-Optimizer/guides/8_autocast.html)
can be used to convert the network to be strongly typed.
--stronglyTyped Create a strongly typed network. (default = d
isabled)
--directIO [Deprecated] Avoid re
formatting at network boundaries. (default = d
isabled)
--prec
isionConstraints=spec Control prec
ision constraint setting. (default = none)
Prec
ision Constraints: spec ::= "none" | "obey" | "prefer"
none = no constraints
prefer = meet prec
ision constraints set by
--layerPrec
isions/
--layerOutputTypes if possible
obey = meet prec
ision constraints set by
--layerPrec
isions/
--layerOutputTypes or fail
otherw
ise
--layerPrec
isions=spec Control per
-layer prec
ision constraints. Effective only when prec
isionConstraints
is set to
"obey" or "prefer". (default = none)
The specs are read left
-to
-right, and later ones override earlier ones. Each layer name can
contain at most one wildcard ('*') character.
Per
-layer prec
ision spec ::= layerPrec
ision[","spec]
layerPrec
ision ::= layerName":"prec
ision
prec
ision ::= "fp32"|"fp16"|"bf16"|"int32"|"int8"
--layerOutputTypes=spec Control per
-layer output type constraints. Effective only when prec
isionConstraints
is set to
"obey" or "prefer". (default = none
The specs are read left
-to
-right, and later ones override earlier ones. Each layer name can
contain at most one wildcard ('*') character. If a layer has more than
one output, then multiple types separated by "+" can be provided
for th
is layer.
Per
-layer output type spec ::= layerOutputTypes[","spec]
layerOutputTypes ::= layerName":"type
type ::= "fp32"|"fp16"|"bf16"|"int32"|"int8"["+"type]
--layerDeviceTypes=spec Specify layer
-specific device type.
The specs are read left
-to
-right, and later ones override earlier ones. If a layer does
not have
a device type specified, the layer will opt
for the default device type.
Per
-layer device type spec ::= layerDeviceTypePair[","spec]
layerDeviceTypePair ::= layerName":"deviceType
deviceType ::= "GPU"|"DLA"
--decomposableAttentions=spec Specify decomposable attentions by comma
-separated names.
The specs are read left
-to
-right, and later ones override earlier ones. Each layer name can
contain at most one wildcard ('*') character.
--calib=<file> Read INT8 calibration
cache file
--safe
Enable build safety certified engine,
--stronglyTyped will be
enabled by default with th
is option.
If DLA
is enabled,
--buildDLAStandalone will be specified
--dumpKernelText Dump the kernel text to a file, only available when
--safe
is enabled
--buildDLAStandalone
Enable build DLA standalone loadable which can be loaded by cuDLA, when th
is option
is enabled,
--allowGPUFallback
is d
isallowed and
--skipInference
is enabled by default. Additionally,
specifying
--inputIO
Formats and
--outputIO
Formats restricts I/O data type and memory layout
(default = d
isabled)
--allowGPUFallback When DLA
is enabled, allow GPU fallback
for unsupported layers (default = d
isabled)
--cons
istency Per
form cons
istency checking on safety certified engine
--restricted
Enable safety scope checking with kSAFETY_SCOPE build flag
--saveEngine=<file> Save the serialized engine
--loadEngine=<file> Load a serialized engine
--asyncFileReader Load a serialized engine using async stream reader. Should be combined with
--loadEngine.
--getPlanVersionOnly Print TensorRT version when loaded plan was created. Works without deserialization of the plan.
Use together with
--loadEngine. Supported only
for engines created with 8.6 and
forward.
--tacticSources=tactics Specify the tactics to be used by adding (+) or removing (
-) tactics from the default
tactic sources (default = all available tactics).
Note: Currently only cuDNN, cuBLAS, cuBLAS
-LT, and edge mask convolutions are l
isted as optional
tactics.
Tactic Sources: tactics ::= tactic[","tactics]
tactic ::= (+|
-)lib
lib ::= "CUBLAS"|"CUBLAS_LT"|"CUDNN"|"EDGE_MASK_CONVOLUTIONS"
|"JIT_CONVOLUTIONS"
For example, to d
isable cudnn and
enable cublas:
--tacticSources=
-CUDNN,+CUBLAS
--noBuilder
Cache D
isable timing
cache in builder (default
is to
enable timing
cache)
--noCompilation
Cache D
isable Compilation
cache in builder, and the
cache is part of timing
cache (default
is to
enable compilation
cache)
--errorOnTiming
CacheM
iss Emit error when a tactic being timed
is not present in the timing
cache (default = false)
--timing
CacheFile=<file> Save/load the serialized global timing
cache
--preview=features Specify preview feature to be used by adding (+) or removing (
-) preview features from the default
Preview Features: features ::= feature[","features]
feature ::= (+|
-)flag
flag ::= "aliasedPluginIO1003"
|"runtimeActivationResize"
|"profileSharing0806"
--builderOptimization
Level Set the builder optimization
level. (default
is 3)
A Higher
level allows TensorRT to spend more time searching
for better optimization strategy.
Valid values include integers from 0 to the maximum optimization
level, which
is currently 5.
--maxTactics Set the maximum number of tactics to time when there
is a choice of tactics. (default
is -1)
Larger number of tactics allow TensorRT to spend more building time on evaluating tactics.
Default value
-1 means TensorRT can decide the number of tactics based on its own heur
istic.
--hardwareCompatibility
Level=mode Make the engine file compatible with other GPU architectures. (default = none)
Hardware Compatibility
Level: mode ::= "none" | "ampere+" | "sameComputeCapability"
none = no compatibility
ampere+ = compatible with Ampere and newer GPUs
sameComputeCapability = compatible with GPUs that have the same Compute Capability version
--runtimePlat
form=plat
form Set the target plat
form
for runtime execution. (default = SameAsBuild)
When th
is option
is enabled,
--skipInference
is enabled by default.
RuntimePlatfrom: plat
form ::= "SameAsBuild" | "WindowsAMD64"
SameAsBuild = no requirement
for cross
-plat
form compatibility.
WindowsAMD64 = set the target plat
form
for engine execution as Windows AMD64 system
--tempdir=<dir> Overrides the default temporary directory TensorRT will use when creating temporary files.
See IRuntime::setTemporaryDirectory API documentation
for more in
formation.
--tempfileControls=controls Controls what TensorRT
is allowed to use when creating temporary executable files.
Should be a comma
-separated l
ist with entries in the
format (in_memory|temporary):(allow|deny).
in_memory: Controls whether TensorRT
is allowed to create temporary in
-memory executable files.
temporary: Controls whether TensorRT
is allowed to create temporary executable files in the
filesystem (in the directory given by
--tempdir).
For example, to allow in
-memory files and d
isallow temporary files:
--tempfileControls=in_memory:allow,temporary:deny
If a flag
is unspecified, the default behavior
is "allow".
--maxAuxStreams=N Set maximum number of auxiliary streams per inference stream that TRT
is allowed to use to run
kernels in parallel if the network contains ops that can run in parallel, with the cost of more
memory
usage. Set th
is to 0
for optimal memory
usage. (default = using heur
istics)
--profile Build with dynamic shapes using a profile with the min/max/opt shapes provided. Can be specified
multiple times to create multiple profiles with contiguous index.
(ex:
--profile=0
--minShapes=<spec>
--optShapes=<spec>
--maxShapes=<spec>
--profile=1 ...)
--calibProfile Select the optimization profile to calibrate by index. (default = 0)
--allowWeightStreaming
Enable a weight streaming engine. Must be specified with
--stronglyTyped. TensorRT will d
isable
weight streaming at runtime unless
--weightStreamingBudget
is specified.
--markDebug Specify l
ist of names of tensors to be marked as debug tensors. Separate names with a comma
--markUnfusedTensorsAsDebugTensors Mark unfused tensors as debug tensors
--tilingOptimization
Level Set the tiling optimization
level. (default
is 0)
A Higher
level allows TensorRT to spend more time searching
for better optimization strategy.
Valid values include integers from 0 to the maximum tiling optimization
level(3).
--l2Limit
ForTiling Set the L2
cache usage limit
for tiling optimization(default
is -1)
--remoteAutoTuningConfig Set the remote auto tuning config. Must be specified with
--safe.
Format: protocol://username[:password]@hostname[:port]?param1=value1¶m2=value2
Example: ssh://user:pass@192.0.2.100:22?remote_exec_path=/opt/tensorrt/bin&remote_lib_path=/opt/tensorrt/lib
--refitFromOnnx Refit the loaded engine with the weights from the provided ONNX model.
The model should be identical to the one used to generate the engine.
=== Inference Options ===
--shapes=spec Set input shapes
for dynamic shapes inference inputs.
Note: Input names can be wrapped with escaped single quotes (ex: 'Input:0').
Example input shapes spec: input0:1x3x256x256, input1:1x3x128x128
For scalars (0
-D shapes), use input0:scalar or simply input0: with
nothing after the colon.
Each input shape
is supplied as a key
-value pair where key
is the input name and
value
is the dimensions (including the batch dimension) to be used
for that input.
Each key
-value pair has the key and value separated using a colon (:).
Multiple input shapes can be provided via comma
-separated key
-value pairs, and each input
name can contain at most one wildcard ('*') character.
--loadInputs=spec Load input values from files (default = generate random inputs). Input names can be wrapped with single quotes (ex: 'Input:0')
Input values spec ::= Ival[","spec]
Ival ::= name":"file
Consult the README
for more in
formation on generating files
for custom inputs.
--iterations=N Run at least N inference iterations (default = 10)
--warmUp=N Run
for N mill
iseconds to warmup be
fore measuring per
formance (default = 200)
--duration=N Run per
formance measurements
for at least N
seconds wallclock time (default = 3)
If
-1
is specified, inference will keep running unless stopped manually
--sleepTime=N Delay inference start with a gap of N mill
iseconds between launch and compute (default = 0)
--idleTime=N Sleep N mill
iseconds between two continuous iterations(default = 0)
--infStreams=N Instantiate N execution contexts to run inference concurrently (default = 1)
--exposeDMA Serialize DMA transfers to and from device (default = d
isabled).
--noDataTransfers D
isable DMA transfers to and from device (default =
enabled).
Note some device
-to
-host
data transfers will remain if output dumping
is enabled via the
--dumpOutput or
--exportOutput flags.
--useManagedMemory Use managed memory instead of separate host and device allocations (default = d
isabled).
--useSpinWait Actively synchronize on GPU events. Th
is option may decrease synchronization time but increase CPU
usage and power (default = d
isabled)
--threads
Enable multithreading to drive engines with independent threads or speed up refitting (default = d
isabled)
--useCudaGraph Use CUDA graph to capture engine execution and then launch inference (default = d
isabled).
Th
is flag may be ignored if the graph capture fails.
--timeDeserialize Time the amount of time it takes to deserialize the network and exit.
--timeRefit Time the amount of time it takes to refit the engine be
fore inference.
--separateProfileRun Do
not attach the profiler in the benchmark run; if profiling
is enabled, a
second profile run will be executed (default = d
isabled)
--skipInference Exit after the engine has been built and skip inference perf measurement (default = d
isabled)
--pers
istent
CacheRatio Set the pers
istent
CacheLimit in ratio, 0.5 represent half of max pers
istent L2 size (default = 0)
--useProfile Set the optimization profile
for the inference context (default = 0 ).
--allocationStrategy=spec Specify how the internal device memory
for inference
is allocated.
Strategy: spec ::= "static"|"profile"|"runtime"
static = Allocate device memory based on max size across all profiles.
profile = Allocate device memory based on max size of the current profile.
runtime = Allocate device memory based on the actual input shapes.
--saveDebugTensors Specify l
ist of names of tensors to turn on the debug state
and filename to save raw outputs to.
These tensors must be specified as debug tensors during build time.
Input values spec ::= Ival[","spec]
Ival ::= name":"file
--saveAllDebugTensors Save all debug tensors to files.
Including debug tensors marked by
--markDebug and
--markUnfusedTensorsAsDebugTensors
Multiple file
formats can be saved simultaneously.
Input values spec ::=
format[","
format]
format ::= "summary"|"numpy"|"string"|"raw"
--weightStreamingBudget Set the maximum amount of GPU memory TensorRT
is allowed to use
for weights.
It can take on the following values:
-2: (default) D
isable weight streaming at runtime.
-1: TensorRT will automatically decide the budget.
0
-100%: Percentage of streamable weights that reside on the GPU.
0% saves the most memory but will have the worst per
formance.
Requires the '%' character.
>=0B: The exact amount of streamable weights that reside on the GPU. Supports the
following base
-2 suffixes: B (Bytes), G (Gibibytes), K (Kibibytes), M (Mebibytes).
=== Reporting Options ===
--verbose Use verbose logging (default = false)
--avgRuns=N Report per
formance measurements averaged over N consecutive iterations (default = 10)
--percentile=P1,P2,P3,... Report per
formance
for the P1,P2,P3,... percentages (0<=P_i<=100, 0 representing max perf, and 100 representing min perf; (default = 90,95,99%)
--dumpRefit Print the refittable layers and weights from a refittable engine
--dumpOutput Print the output tensor(s) of the last inference iteration (default = d
isabled)
--dumpRawBindingsToFile Print the input/output tensor(s) of the last inference iteration to file(default = d
isabled)
--dumpProfile Print profile in
formation per layer (default = d
isabled)
--dumpLayerInfo Print layer in
formation of the engine to console (default = d
isabled)
--dumpOptimizationProfile Print the optimization profile(s) in
formation (default = d
isabled)
--exportTimes=<file> Write the timing results in a json file (default = d
isabled)
--exportOutput=<file> Write the output tensors to a json file (default = d
isabled)
--exportProfile=<file> Write the profile in
formation per layer in a json file (default = d
isabled)
--exportLayerInfo=<file> Write the layer in
formation of the engine in a json file (default = d
isabled)
=== System Options ===
--device=N Select cuda device N (default = 0)
--useDLACore=N Select DLA core N
for layers that support DLA (default = none)
--staticPlugins Plugin library (.so) to load statically (can be specified multiple times)
--dynamicPlugins Plugin library (.so) to load dynamically and may be serialized with the engine if they are included in
--setPluginsToSerialize (can be specified multiple times)
--setPluginsToSerialize Plugin library (.so) to be serialized with the engine (can be specified multiple times)
--ignoreParsedPluginLibs By default, when building a version
-compatible engine, plugin libraries specified by the ONNX parser
are implicitly serialized with the engine (unless
--excludeLeanRuntime
is specified) and loaded dynamically.
Enable th
is flag to ignore these plugin libraries instead.
--safetyPlugins Plugin library (.so)
for TensorRT auto safety to manually load safety plugins specified by the command line arguments.
Example:
--safetyPlugins=/path/to/plugin_lib.so[pluginNamespace1::plugin1,pluginNamespace2::plugin2].
The option can be specified multiple times with different plugin libraries.
=== Help ===
--help,
-h Print th
is message
[11/27/2025
-14:33:27] [E] Model m
issing or
format
not recognized
&&&& FAILED TensorRT.trtexec [TensorRT v101401] [b48] # trtexec
--version
本文解决了一个关于Hibernate二级缓存未启用的问题。在项目开发过程中，由于未配置缓存导致出现异常。通过手动配置EhCache来启用二级缓存，解决了这个问题。
扫一扫
1147
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
