Avoiding Memory Leaks

Avoiding Memory Leaks


Avoiding Memory Leaks

Android applications are, at least on the T-Mobile G1, limited to 16 MB of heap. It's both a lot of memory for a phone and yet very little for what some developers want to achieve. Even if you do not plan on using all of this memory, you should use as little as possible to let other applications run without getting them killed. The more applications Android can keep in memory, the faster it will be for the user to switch between his apps. As part of my job, I ran into memory leaks issues in Android applications and they are most of the time due to the same mistake: keeping a long-lived reference to a Context.

On Android, a Context is used for many operations but mostly to load and access resources. This is why all the widgets receive a Context parameter in their constructor. In a regular Android application, you usually have two kinds of ContextActivity and Application. It's usually the first one that the developer passes to classes and methods that need a Context:

@Override
protected void onCreate(Bundle state) {
  super.onCreate(state);
  
  TextView label = new TextView(this);
  label.setText("Leaks are bad");
  
  setContentView(label);
}

This means that views have a reference to the entire activity and therefore to anything your activity is holding onto; usually the entire View hierarchy and all its resources. Therefore, if you leak the Context("leak" meaning you keep a reference to it thus preventing the GC from collecting it), you leak a lot of memory. Leaking an entire activity can be really easy if you're not careful.

When the screen orientation changes the system will, by default, destroy the current activity and create a new one while preserving its state. In doing so, Android will reload the application's UI from the resources. Now imagine you wrote an application with a large bitmap that you don't want to load on every rotation. The easiest way to keep it around and not having to reload it on every rotation is to keep in a static field:

private static Drawable sBackground;
  
@Override
protected void onCreate(Bundle state) {
  super.onCreate(state);
  
  TextView label = new TextView(this);
  label.setText("Leaks are bad");
  
  if (sBackground == null) {
    sBackground = getDrawable(R.drawable.large_bitmap);
  }
  label.setBackgroundDrawable(sBackground);
  
  setContentView(label);
}

This code is very fast and also very wrong; it leaks the first activity created upon the first screen orientation change. When a Drawable is attached to a view, the view is set as a callback on the drawable. In the code snippet above, this means the drawable has a reference to the TextView which itself has a reference to the activity (the Context) which in turns has references to pretty much anything (depending on your code.)

This example is one of the simplest cases of leaking the Context and you can see how we worked around it in the Home screen's source code (look for the unbindDrawables() method) by setting the stored drawables' callbacks to null when the activity is destroyed. Interestingly enough, there are cases where you can create a chain of leaked contexts, and they are bad. They make you run out of memory rather quickly.

There are two easy ways to avoid context-related memory leaks. The most obvious one is to avoid escaping the context outside of its own scope. The example above showed the case of a static reference but inner classes and their implicit reference to the outer class can be equally dangerous. The second solution is to use the Application context. This context will live as long as your application is alive and does not depend on the activities life cycle. If you plan on keeping long-lived objects that need a context, remember the application object. You can obtain it easily by callingContext.getApplicationContext() or Activity.getApplication().

In summary, to avoid context-related memory leaks, remember the following:

  • Do not keep long-lived references to a context-activity (a reference to an activity should have the same life cycle as the activity itself)
  • Try using the context-application instead of a context-activity
  • Avoid non-static inner classes in an activity if you don't control their life cycle, use a static inner class and make a weak reference to the activity inside. The solution to this issue is to use a static inner class with a WeakReference to the outer class, as done in ViewRoot and its W inner class for instance
  • A garbage collector is not an insurance against memory leaks

在深度学习训练过程中,`DistributedDataParallel`(DDP)是常用的分布式训练工具,它可以在多个GPU上并行训练模型。然而,DDP在某些情况下可能会增加内存使用量,特别是在小批量(batch size)训练时。为了减少内存使用量,可以考虑以下几种方法: 1. **使用`DataParallel`代替`DistributedDataParallel`**:`DataParallel`是另一种数据并行工具,它在单台机器上使用多个GPU进行并行计算。相比DDP,`DataParallel`的实现更简单,内存开销也更小。 2. **减少模型参数量**:通过减少模型的层数或每层的神经元数量,可以有效减少内存使用量。 3. **使用混合精度训练**:混合精度训练可以在保持模型精度的情况下减少内存使用量。常用的方法是使用`torch.cuda.amp`进行自动混合精度训练。 4. **梯度累积**:通过累积多个小批量数据的梯度再进行一次反向传播更新模型参数,可以减少内存使用量。常用的方法是使用`torch.utils.data.DataLoader`的`accumulate_grad_batches`参数。 5. **优化数据结构**:使用更节省内存的数据结构,如`torch.utils.checkpoint`可以在反向传播时重新计算中间结果,而不是存储它们。 以下是使用`DataParallel`代替`DistributedDataParallel`的一个简单示例: ```python import torch import torch.nn as nn from torch.nn.parallel import DataParallel # 假设我们有一个简单的模型 class SimpleModel(nn.Module): def __init__(self): super(SimpleModel, self).__init__() self.layer = nn.Linear(10, 10) def forward(self, x): return self.layer(x) # 初始化模型 model = SimpleModel() # 使用DataParallel进行并行计算 model = DataParallel(model) # 假设我们有一个输入张量 input_tensor = torch.randn(64, 10) # 前向传播 output = model(input_tensor) # 反向传播 output.mean().backward() ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值