tf.expand_dims()和 tf.reshape()的区别

TensorFlow维度扩展
本文介绍在TensorFlow中如何使用tf.expand_dims()函数增加张量维度,对比tf.reshape()的局限性,尤其在处理未确定形状的占位符时。通过实例演示如何在图像处理中正确应用该函数。
部署运行你感兴趣的模型镜像

 

TensorFlow中,想要维度增加一维,可以使用tf.expand_dims(input, dim, name=None)函数。当然,我们常用tf.reshape(input, shape=[])也可以达到相同效果,但是有些时候在构建图的过程中,placeholder没有被feed具体的值,这时就会包下面的错误:

TypeError: Expected binary or unicode string, got 1 


在这种情况下,我们就可以考虑使用expand_dims来将维度加1。比如我自己代码中遇到的情况,在对图像维度降到二维做特定操作后,要还原成四维[batch, height, width, channels],前后各增加一维。如果用reshape,则因为上述原因报错

one_img2 = tf.reshape(one_img, shape=[1, one_img.get_shape()[0].value, one_img.get_shape()[1].value, 1])

用下面的方法可以实现:

one_img = tf.expand_dims(one_img, 0)
one_img = tf.expand_dims(one_img, -1) #-1表示最后一维

在最后,给出官方的例子和说明

 

# 't' is a tensor of shape [2]
shape(expand_dims(t, 0)) ==> [1, 2]
shape(expand_dims(t, 1)) ==> [2, 1]
shape(expand_dims(t, -1)) ==> [2, 1]

# 't2' is a tensor of shape [2, 3, 5]
shape(expand_dims(t2, 0)) ==> [1, 2, 3, 5]
shape(expand_dims(t2, 2)) ==> [2, 3, 1, 5]
shape(expand_dims(t2, 3)) ==> [2, 3, 5, 1]

 

Args: 
input: A Tensor. 
dim: A Tensor. Must be one of the following types: int32, int64. 0-D (scalar). Specifies the dimension index at which to expand the shape of input. 
name: A name for the operation (optional).

Returns: 
A Tensor. Has the same type as input. Contains the same data as input, but its shape has an additional dimension of size 1 added.

 

转载:https://blog.youkuaiyun.com/jasonzzj/article/details/60811035

您可能感兴趣的与本文相关的镜像

TensorFlow-v2.15

TensorFlow-v2.15

TensorFlow

TensorFlow 是由Google Brain 团队开发的开源机器学习框架,广泛应用于深度学习研究和生产环境。 它提供了一个灵活的平台,用于构建和训练各种机器学习模型

class MLA(layers.Layer): def __init__(self, args: ModelArgs): super().__init__() self.dim = args.dim self.n_heads = args.n_heads self.q_lora_rank = args.q_lora_rank self.kv_lora_rank = args.kv_lora_rank self.qk_nope_head_dim = args.qk_nope_head_dim self.qk_rope_head_dim = args.qk_rope_head_dim self.qk_head_dim = args.qk_nope_head_dim + args.qk_rope_head_dim self.v_head_dim = args.v_head_dim # 初始化投影层 if self.q_lora_rank == 0: self.wq = layers.Dense(self.n_heads * self.qk_head_dim) else: self.wq_a = layers.Dense(self.q_lora_rank) self.q_norm = RMSNorm(self.q_lora_rank) self.wq_b = layers.Dense(self.n_heads * self.qk_head_dim) self.wkv_a = layers.Dense(self.kv_lora_rank + self.qk_rope_head_dim) self.kv_norm = RMSNorm(self.kv_lora_rank) self.wkv_b = layers.Dense(self.n_heads * (self.qk_nope_head_dim + self.v_head_dim)) self.wo = layers.Dense(self.dim) self.softmax_scale = self.qk_head_dim ** -0.5 if args.max_seq_len > args.original_seq_len: mscale = 0.1 * args.mscale * math.log(args.rope_factor) + 1.0 self.softmax_scale *= mscale * mscale # 初始化缓存 self.k_cache = tf.Variable(tf.zeros((args.max_batch_size, args.max_seq_len,self.n_heads, self.qk_head_dim)),trainable=False) self.v_cache = tf.Variable(tf.zeros((args.max_batch_size, args.max_seq_len,self.n_heads, self.v_head_dim)),trainable=False) def call(self, x, start_pos, freqs_cis, mask=None): bsz = tf.shape(x)[0] seqlen = tf.shape(x)[1] end_pos = start_pos + seqlen # 查询投影 if self.q_lora_rank == 0: q = self.wq(x) else: q = self.wq_b(self.q_norm(self.wq_a(x))) q = tf.reshape(q, [bsz, seqlen, self.n_heads, self.qk_head_dim]) q_nope, q_pe = tf.split(q, [self.qk_nope_head_dim, self.qk_rope_head_dim], axis=-1) q_pe = apply_rotary_emb(q_pe, freqs_cis) # 键值投影 kv = self.wkv_a(x) kv, k_pe = tf.split(kv, [self.kv_lora_rank, self.qk_rope_head_dim], axis=-1) k_pe = apply_rotary_emb(tf.expand_dims(k_pe, 2), freqs_cis) kv = self.wkv_b(self.kv_norm(kv)) kv = tf.reshape(kv, [bsz, seqlen, self.n_heads, self.qk_nope_head_dim + self.v_head_dim]) k_nope, v = tf.split(kv, [self.qk_nope_head_dim, self.v_head_dim], axis=-1) k = tf.concat([k_nope, tf.tile(k_pe, [1, 1, self.n_heads, 1])], axis=-1) # 更新缓存 updates_range = tf.range(start_pos, end_pos) self.k_cache.assign(tf.tensor_scatter_nd_update(self.k_cache,updates_range[:, None],k)) self.v_cache.assign(tf.tensor_scatter_nd_update(self.v_cache,updates_range[:, None],v)) # 注意力计算 q = tf.concat([q_nope, q_pe], axis=-1) scores = tf.einsum("bshd,bthd->bhst", q, self.k_cache[:bsz, :end_pos]) * self.softmax_scale if mask is not None: scores += mask[:, None, :, :] scores = tf.nn.softmax(scores, axis=-1) x = tf.einsum("bhst,bthd->bshd", scores, self.v_cache[:bsz, :end_pos]) return self.wo(tf.reshape(x, [bsz, seqlen, -1])) 将缓存去掉
03-13
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值