【机器学习系列（11）】AI基础技术与轻量化实践-优快云博客

本文链接：https://blog.youkuaiyun.com/Azperk/article/details/146280838

【机器学习系列（11）】AI基础技术与轻量化实践

一、强化学习基础

1. Q-Learning算法原理

$\leftarrow Q(s,a) + \alpha [r + \gamma \max_{a'} Q(s',a') - Q(s,a)]$

2. CartPole平衡实战

import gym
import numpy as np

# 创建Q表（状态离散化）
state_bins = [np.linspace(-4.8, 4.8, 4),  # 车的位置
              np.linspace(-5, 5, 4)]      # 车速度

def get_state(observation):
    return tuple(np.digitize(obs, bin) for obs, bin in zip(observation, state_bins))

env = gym.make('CartPole-v1')
q_table = np.zeros((4, 4, 2))  # 状态数×动作数

# 训练循环
for episode in range(1000):
    obs = env.reset()
    state = get_state(obs)
    while True:
        action = np.argmax(q_table[state]) 
        next_obs, reward, done, _ = env.step(action)
        next_state = get_state(next_obs)
        # Q值更新
        q_table[state][action] += 0.1 * (reward + 0.9 * np.max(q_table[next_state]) - q_table[state][action])
        state = next_state
        if done: break

二、自监督学习入门

1. 旋转预测预训练

from torchvision import transforms

class RotationTransform:
    def __call__(self, x):
        angle = np.random.choice([0, 90, 180, 270])
        return transforms.functional.rotate(x, angle)

model = torch.nn.Sequential(
    torch.nn.Conv2d(3, 16, 3),
    torch.nn.MaxPool2d(2),
    torch.nn.Linear(16*111*111, 4)  # 预测旋转角度分类
)

# 损失函数（四分类交叉熵）
loss_fn = torch.nn.CrossEntropyLoss()

三、边缘AI部署实战

1. 模型蒸馏示例

teacher_model = torch.load('resnet18.pth')  # 预训练教师模型
student_model = torch.nn.Sequential(        # 轻量学生模型
    torch.nn.Conv2d(3, 8, 3),
    torch.nn.Flatten(),
    torch.nn.Linear(8*222*222, 10)
)

# 蒸馏损失
def distillation_loss(student_out, teacher_out, T=2): 
    return torch.nn.KLDivLoss()(
        torch.log_softmax(student_out/T, dim=1),
        torch.softmax(teacher_out/T, dim=1)
    )