CARLA平台+Q-learning的尝试（gym-carla）

接触强化学习大概有半年了，也了解了一些算法，一些简单的算法在gym框架也实现了，那么结合仿真平台Carla该怎么用呢？由于比较熟悉gym框架，就偷个懒先从这个开始写代码。

项目地址：https://github.com/cjy1992/gym-carla

一、环境配置

1.1 基本配置

• Ubuntu 16.04（作者在Ubuntu 20.04测试成功）
• Anaconda
• Carla 0.96

1.2 配置工作环境

• 建立pyhon3.6的环境

$ conda create -n py36 python=3.6

• git代码

$ git clone https://github.com/cjy1992/gym-carla.git

• 进入该路径，cd XXX

$ pip install -r requiremtns.txt
$ pip install -e .

• 下载Carla0.9.6版本

  https://github.com/carla-simulator/carla/releases/tag/0.9.6

• 添加环境变量（否则会出现No module named 'carla'）

  • 方法一：

    $ conda activate py36
    $ export PYTHONPATH=$PYTHONPATH:/home/shy/CARLA_0.9.6/PythonAPI/carla/dist/carla-0.9.6-py3.5-linux-x86_64.egg
    

  • 方法二：在环境变量配置文件中添加上述环境变量

  • 方法三：安装easy_install后，easy_install XXX.egg即可

  • 方法四：在主函数代码里添加这句即可

    import sys
    try:
        sys.path.append('/home/shy/CARLA_0.9.6/PythonAPI/carla/dist/carla-0.9.6-py3.5-linux-x86_64.egg')
    

    方法五：https://github.com/carla-simulator/carla/issues/1466

1.3 运行测试

• 打开CARLA的目录，右键终端启动Carla

$ ./CarlaUE4.sh -windowed -carla-port=2000

也可以启动无界面模式（提高运行速度）

$ DISPLAY= ./CarlaUE4.sh -opengl -carla-port=2000

• 打开该项目的目录，右键终端输入python test.py

二、环境解读

2.1 test.py–超参数设置

#!/usr/bin/env python

# Copyright (c) 2019: Jianyu Chen (jianyuchen@berkeley.edu).
#
# This work is licensed under the terms of the MIT license.
# For a copy, see <https://opensource.org/licenses/MIT>.

import gym
import sys
try:
    sys.path.append('/home/shy/CARLA_0.9.6/PythonAPI/carla/dist/carla-0.9.6-py3.5-linux-x86_64.egg') # 手动添加环境变量
except IndexError:
    pass
import gym_carla
import carla

def main():
  # parameters for the gym_carla environment
  params = {
   
   
    'number_of_vehicles': 100,
    'number_of_walkers': 0,
    'display_size': 512,  # screen size of bird-eye render
    'max_past_step': 1,  # the number of past steps to draw
    'dt': 0.1,  # time interval between two frames
    'discrete': False,  # whether to use discrete control space
    'discrete_acc': [-3.0, 0.0, 3.0],  # discrete value of accelerations
    'discrete_steer': [-0.2, 0.0, 0.2],  # discrete value of steering angles
    'continuous_accel_range': [-3.0, 3.0],  # continuous acceleration range
    'continuous_steer_range': [-0.3, 0.3],  # continuous steering angle range
    'ego_vehicle_filter': 'vehicle.lincoln*',  # filter for defining ego vehicle
    'port': 2000,  # connection port
    'town': 'Town03',  # which town to simulate
    'task_mode': 'random',  # mode of the task, [random, roundabout (only for Town03)]
    'max_time_episode': 1000,  # maximum timesteps per episode
    'max_waypt': 12,  # maximum number of waypoints
    'obs_range': 32,  # observation range (meter)
    'lidar_bin': 0.125,  # bin size of lidar sensor (meter)
    'd_behind': 12,  # distance behind the ego vehicle (meter)
    'out_lane_thres': 2.0,  # threshold for out of lane
    'desired_speed': 30,  # desired speed (m/s)
    'max_ego_spawn_times': 200,  # maximum times to spawn ego vehicle
    'display_route': True,  # whether to render the desired route