手推遗传算法,寻找最大值(单目标函数和多目标函数)

本文介绍了如何使用遗传算法求解单个参数的最大值,以及扩展到多目标优化的情况,通过定义适应度函数、初始化种群、选择、突变和交叉等步骤进行迭代优化。

单目标函数

# 通过遗传算法求解单个参数的最大值
import random


# 定义适应度函数
def fitness_function(x):
    return x ** 2 - 4 * x + 4


# 初始化总群
def initialize_population(pop_size, lower_bound, upper_bound):
    return [random.uniform(lower_bound, upper_bound) for _ in range(pop_size)]


# 计算每个总群的适应度
def calculate_fitness(population):
    return [fitness_function(individual) for individual in population]


# 选择:根据他们的适应度选择最佳种群
def selection(population, fitness_values, num_parents):
    sorted_population = [x for _, x in sorted(zip(fitness_values, population), reverse=True)]
    return sorted_population[:num_parents]


# 突变:引入随机突变以保持多样性
def mutation(individual, mutation_rate, lower_bound, upper_bound):
    if random.random() < mutation_rate:
        return random.uniform(lower_bound, upper_bound)
    return individual


# 遗传算法函数
def genetic_algorithm(pop_size, num_generations, num_parents, mutation_rate, lower_bound, upper_bound):
    population = initialize_population(pop_size, lower_bound, upper_bound)

    for generation in range(num_generations):
        fitness_values = calculate_fitness(population)
        parents = selection(population, fitness_values, num_parents)

        new_population = []

        for parent in parents:
            child = mutation(parent, mutation_rate, lower_bound, upper_bound)
            new_population.append(child)

        population = new_population

    best_individual = max(population, key=fitness_function)
    best_fitness = fitness_function(best_individual)

    return best_individual, best_fitness


if __name__ == "__main__":
    pop_size = 50
    num_generations = 100
    num_parents = 20
    mutation_rate = 0.1
    lower_bound = -10
    upper_bound = 10

    best_individual, best_fitness = genetic_algorithm(pop_size, num_generations, num_parents, mutation_rate,
                                                      lower_bound, upper_bound)

    print("Best individual:", best_individual)
    print("Best fitness:", best_fitness)

多目标函数

import random
from random import sample

# 定义要优化的函数(假设有五个参数 x1, x2, x3, x4, x5)
def fitness_function(x1, x2, x3, x4, x5):
    return -(x1 + 2*x2 + 3*x3 + 4*x4 + 5*x5)  # 负号用于寻找最大值

# 初始化种群
def initialize_population(pop_size, lower_bound, upper_bound):
    return [[random.uniform(lower_bound, upper_bound) for _ in range(5)] for _ in range(pop_size)]

# 计算适应度
def calculate_fitness(population):
    return [fitness_function(*individual) for individual in population]

# 选择
def selection(population, fitness_values, num_parents):
    sorted_population = [x for _, x in sorted(zip(fitness_values, population), reverse=True)]
    return sorted_population[:num_parents]
    # return sorted_population[:num_parents], sorted_population[num_parents:]

# 交叉
def crossover(parent1, parent2):
    crossover_point = random.randint(1, 4)
    child1 = parent1[:crossover_point] + parent2[crossover_point:]
    child2 = parent2[:crossover_point] + parent1[crossover_point:]
    return child1, child2

# 变异
def mutation(individual, mutation_rate, lower_bound, upper_bound):
    for i in range(5):
        if random.random() < mutation_rate:
            individual[i] = random.uniform(lower_bound, upper_bound)
    return individual

# 遗传算法函数
def genetic_algorithm(pop_size, num_generations, num_parents, mutation_rate, lower_bound, upper_bound):
    population = initialize_population(pop_size, lower_bound, upper_bound)

    for generation in range(num_generations):
        fitness_values = calculate_fitness(population)
        parents = selection(population, fitness_values, num_parents)
        # parents, rest = selection(population, fitness_values, num_parents)
        new_population = []

        for i in range(0, len(parents), 2):
            parent1, parent2 = parents[i], parents[i+1]
            child1, child2 = crossover(parent1, parent2)
            child1 = mutation(child1, mutation_rate, lower_bound, upper_bound)
            child2 = mutation(child2, mutation_rate, lower_bound, upper_bound)
            new_population.extend([child1, child2])

        # population = new_population + rest
        population = new_population + sample(population, pop_size-num_parents)

        # while len(new_population) < pop_size:  # Keep generating children until reaching the desired pop_size
        #     parent1, parent2 = random.sample(parents, 2)
        #     child1, child2 = crossover(parent1, parent2)
        #     child1 = mutation(child1, mutation_rate, lower_bound, upper_bound)
        #     child2 = mutation(child2, mutation_rate, lower_bound, upper_bound)
        #     new_population.extend([child1, child2])
        #
        # population = new_population[:pop_size]  # Truncate if new_population exceeds pop_size

    best_individual = max(population, key=lambda ind: fitness_function(*ind))
    best_fitness = fitness_function(*best_individual)

    return best_individual, best_fitness

if __name__ == "__main__":
    pop_size = 50
    num_generations = 100
    num_parents = 10
    mutation_rate = 0.1
    lower_bound = -10
    upper_bound = 10

    best_individual, best_fitness = genetic_algorithm(pop_size, num_generations, num_parents, mutation_rate, lower_bound, upper_bound)

    print("Best individual:", best_individual)
    print("Best fitness:", best_fitness)
<think>嗯,用户这次明确要求查找Adaboost算法的手动推导过程。从对话历史看,用户之前提供的参考引用中提到了Adaboost的权重特性(样本权重D特征权重W),但核心需求是推导过程而非概念对比。 用户可能正在学习集成算法,需要从数学层面理解Adaboost的权重更新机制。考虑到算法推导涉及迭代公式,准备重点拆解三个核心步骤:初始化权重、弱分类器训练、权重更新规则,并补充关键公式的数学解释。 注意到用户提供的引用[1]提到“加权求决定结果”,这印证了推导中需要强调分类器权重α_t的计算逻辑。而引用[2]的因果推理内容与当前需求无关,可以忽略。 推导过程需要特别注意两点:一是样本权重归一化的处理(如公式$\sum_{i=1}^n D_t(i)=1$),二是误差率ϵ_t与分类器权重α_t的反比关系(通过$α_t = \frac{1}{2}\ln(\frac{1-ϵ_t}{ϵ_t})$体现)。对于指数损失函数,计划用$H(x) = sign(\sum_{t=1}^T α_th_t(x))$说明加权投票机制。 用户可能是工程背景的研究者,因此会在最后补充手动实现建议(如分类器权重列表维护)迭代停止条件等实用细节。相关问题设计也侧重推导延伸(前向分步算法)而非概念对比,避免重复已有引用内容。 </think>以下是Adaboost算法的手动推导过程详解(以二分类问题为例): --- ### **Adaboost推导核心步骤** #### **1. 初始化样本权重** 设训练集 $\( \{(x_1,y_1), (x_2,y_2), ..., (x_n,y_n)\} \)$,其中 $\( y_i \in \{-1, +1\} \)$。 初始样本权重均匀分布: $$ D_1(i) = \frac{1}{n}, \quad i=1,2,...,n $$ #### **2. 迭代训练弱分类器(共T轮)** **For t=1 to T:** **(a) 训练弱分类器** 使用当前权重 $\( D_t \)$ 训练弱分类器 $\( h_t(x) \)$,目标是最小化加权错误率: $$ \epsilon_t = P_{i \sim D_t} [h_t(x_i) \neq y_i] = \sum_{i: h_t(x_i) \neq y_i} D_t(i) $$ **(b) 计算分类器权重** 根据错误率 $\( \epsilon_t \)$ 计算当前分类器的权重 $\( \alpha_t \)$: $$ \alpha_t = \frac{1}{2} \ln \left( \frac{1 - \epsilon_t}{\epsilon_t} \right) $$ > 推导说明:$\alpha_t$ 是损失函数(指数损失)最小化的解,当 $\epsilon_t < 0.5$ 时 $\alpha_t > 0$;错误率越低,权重越大。 **(c) 更新样本权重** 更新下一轮样本权重 $\( D_{t+1} \)$: $$ D_{t+1}(i) = \frac{D_t(i) \exp \left( -\alpha_t y_i h_t(x_i) \right)}{Z_t} $$ 其中 $\( Z_t \)$ 是归一化因子: $$ Z_t = \sum_{i=1}^n D_t(i) \exp \left( -\alpha_t y_i h_t(x_i) \right) $$ > **权重更新逻辑**: > - 若样本分类正确($y_i h_t(x_i) = +1$),则 $\exp(-\alpha_t) < 1$,权重降低; > - 若分类错误($y_i h_t(x_i) = -1$),则 $\exp(\alpha_t) > 1$,权重升高。 #### **3. 组合强分类器** 最终模型为弱分类器的加权投票: $$ H(x) = \text{sign} \left( \sum_{t=1}^T \alpha_t h_t(x) \right) $$ --- ### **关键公式推导(指数损失最小化)** Adaboost本质是优化指数损失函数: $$ L(H) = \frac{1}{n} \sum_{i=1}^n \exp \left( -y_i H(x_i) \right) $$ 其中 $\( H(x) = \sum_{t} \alpha_t h_t(x) \)$。 通过**前向分步算法**逐轮优化: 1. 假设第 $t$ 轮已得到 $H_{t-1}(x)$; 2. 新增弱分类器 $h_t(x)$ 时,优化目标为: $$ (\alpha_t, h_t) = \arg \min_{\alpha, h} \sum_{i=1}^n \exp \left( -y_i [H_{t-1}(x_i) + \alpha h(x_i)] \right) $$ 3. 推导可得最优 $\alpha_t$ 为前述公式,样本权重更新由损失函数的梯度决定[^1]。 --- ### **手动实现要点** ```python import numpy as np class AdaBoost: def __init__(self, n_estimators=50): self.n_estimators = n_estimators self.alphas = [] # 分类器权重列表 self.classifiers = [] # 弱分类器列表 def fit(self, X, y): n = X.shape[0] D = np.ones(n) / n # 初始化样本权重 for t in range(self.n_estimators): # 1. 训练弱分类器(例如决策树桩) h_t = self._train_weak_classifier(X, y, D) # 2. 计算加权错误率 pred = h_t.predict(X) err = D[(pred != y)].sum() # 3. 计算分类器权重 alpha_t = 0.5 * np.log((1 - err) / max(err, 1e-10)) # 避免除零 # 4. 更新样本权重 D *= np.exp(-alpha_t * y * pred) D /= D.sum() # 归一化 # 存储结果 self.alphas.append(alpha_t) self.classifiers.append(h_t) def predict(self, X): score = np.zeros(X.shape[0]) for alpha, h in zip(self.alphas, self.classifiers): score += alpha * h.predict(X) return np.sign(score) ``` --- **注意事项**: 1. 弱分类器需支持**加权训练**(如 `sklearn.tree.DecisionTreeClassifier(max_depth=1)`); 2. 迭代停止条件可以是 $\epsilon_t \geq 0.5$ 或达到预设轮数; 3. 样本权重更新可能导致数值溢出,需加入平滑项[^1]。 ---
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值