探索Amazon SageMaker Experiments：轻松跟踪您的AI实验！-优快云博客

# 探索Amazon SageMaker Experiments：轻松跟踪您的AI实验！

## 引言

在机器学习领域，实验跟踪和管理是开发和优化模型过程中不可或缺的一部分。Amazon SageMaker Experiments是一项强大的工具，它可以帮助开发者组织、跟踪、比较和评估机器学习实验和模型版本。在这篇文章中，我们将探讨如何使用LangChain Callback将提示和其他超参数记录到SageMaker Experiments中。我们将通过三个不同的场景来展示这种能力。

## 主要内容

### 安装与设置

首先，我们需要安装必要的库并设置API密钥。

```bash
%pip install --upgrade --quiet sagemaker
%pip install --upgrade --quiet langchain-openai
%pip install --upgrade --quiet google-search-results

接下来，设置所需的API密钥：

import os

# 添加你的API密钥
os.environ["OPENAI_API_KEY"] = "<ADD-KEY-HERE>"
os.environ["SERPAPI_API_KEY"] = "<ADD-KEY-HERE>"

场景1：单一语言模型

在这个场景中，我们使用单一LLM模型根据给定的提示生成输出。

# LLM超参数
HPARAMS = {
    "temperature": 0.1,
    "model_name": "gpt-3.5-turbo-instruct",
}

# 使用API代理服务提高访问稳定性
BUCKET_NAME = None
EXPERIMENT_NAME = "langchain-sagemaker-tracker"

# 创建SageMaker会话
session = Session(default_bucket=BUCKET_NAME)

RUN_NAME = "run-scenario-1"
PROMPT_TEMPLATE = "tell me a joke about {topic}"
INPUT_VARIABLES = {"topic": "fish"}

with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:
    sagemaker_callback = SageMakerCallbackHandler(run)

    # 定义LLM模型
    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)

    # 创建提示模板
    prompt = PromptTemplate.from_template(template=PROMPT_TEMPLATE)

    # 创建LLM链
    chain = LLMChain(llm=llm, prompt=prompt, callbacks=[sagemaker_callback])

    # 运行链
    chain.run(**INPUT_VARIABLES)

    sagemaker_callback.flush_tracker()

场景2：顺序链

这里我们创建一个顺序链，使用两个LLM模型。

RUN_NAME = "run-scenario-2"

PROMPT_TEMPLATE_1 = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
Title: {title}
Playwright: This is a synopsis for the above play:"""
PROMPT_TEMPLATE_2 = """You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.
Play Synopsis: {synopsis}
Review from a New York Times play critic of the above play:"""

INPUT_VARIABLES = {
    "input": "documentary about good video games that push the boundary of game design"
}

with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:
    sagemaker_callback = SageMakerCallbackHandler(run)

    # 创建提示模板
    prompt_template1 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_1)
    prompt_template2 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_2)

    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)

    chain1 = LLMChain(llm=llm, prompt=prompt_template1, callbacks=[sagemaker_callback])
    chain2 = LLMChain(llm=llm, prompt=prompt_template2, callbacks=[sagemaker_callback])

    overall_chain = SimpleSequentialChain(chains=[chain1, chain2], callbacks=[sagemaker_callback])
    overall_chain.run(**INPUT_VARIABLES)

    sagemaker_callback.flush_tracker()

场景3：带有工具的代理

在这种情况下，除了LLM，我们还使用多个工具（搜索和数学）。

RUN_NAME = "run-scenario-3"
PROMPT_TEMPLATE = "Who is the oldest person alive? And what is their current age raised to the power of 1.51?"

with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:
    sagemaker_callback = SageMakerCallbackHandler(run)

    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)

    tools = load_tools(["serpapi", "llm-math"], llm=llm, callbacks=[sagemaker_callback])

    agent = initialize_agent(tools, llm, agent="zero-shot-react-description", callbacks=[sagemaker_callback])
    agent.run(input=PROMPT_TEMPLATE)

    sagemaker_callback.flush_tracker()