Skip to content
/ DRL Public

Repository for codes of 'Deep Reinforcement Learning'

Notifications You must be signed in to change notification settings

Kyushik/DRL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DRL

Introduction

This repository is the codes for Deep Reinforcement Learning
I verified my codes with games. The games are made with pygame. I made the games or I modified them to apply for DRL.

Also, environments, which are made by Unity ML-agents, are in Unity_ML_Agent Repository


Performance of each algorithm that I implemented are as follows (verified with Breakout).

Performance

Result

Youtube Link of Playing Breakout with NoisyNet DQN

I set up the DRL code as follows.

  1. Human-level Control Through Deep Reinforcement Learning (DQN)
  2. Deep Reinforcement Learning with Double Q-Learning (DDQN)
  3. Prioritized Experience Replay (PER)
  4. Dueling Network Architecture for Deep Reinforcement Learning (Dueling DQN)
  5. Noisy Networks for Exploration (NoisyNet DQN)
  6. A Distributional Perspective on Reinforcement Learning (C51)
  7. Learning to Predict by the Methods of Temporal Differences (N-step DQN)
  8. Deep Recurrent Q-Learning for Partially Observable MDPs (DRQN)
  9. Distributional Reinforcement Learning with Quantile Regression(QR-DQN)
  10. Implicit Quantile Networks for Distributional Reinforcement Learning(IQN)
  11. Curiosity-driven Exploration by Self-supervised Prediction (ICM)
  12. Exploration by Random Network Distillation(RND)

This is the PPT file for the description of DQN codes that I implemented.

The description of DRL algorithms

Description of algorithms link



Environment

Software

  • Windows7 (64bit)
  • Python 3.6.5
  • Anaconda 4.2.0
  • Tensorflow-gpu 1.12.0
  • pygame 1.9.3
  • opencv3 3.1.0

Hardware

  • CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHZ

  • GPU: GeForce GTX 1080TI

  • Memory: 16GB



Implementation

The sample code for my environment is Environment_sample.py

# This is sample code for Deep Reinforcement Learning testing environment 

# Import modules
import sys 
import numpy as np
import random

# Import games
sys.path.append("DQN_GAMES/")

# add as game the one that you want to play!! 
import pong as game
import dot  
import dot_test  
import tetris  
import wormy
import breakout

# Get Number of action and name of game from the game code
Num_action = game.Return_Num_Action()
game_name = game.ReturnName()

# Get game state class from game code
game_state = game.GameState()

while True:
    # Choose random action
    action = np.zeros([Num_action])
    action[random.randint(0, Num_action - 1)] = 1.0

    # You can get next observation, reward and terminal after action
    observation_next, reward, terminal = game_state.frame_step(action)

You can run the game with random action using this code!!

Please check that you made the folder for saved variables

checkpoint = tf.train.get_checkpoint_state("saved_networks_DQN")

Then you are ready to implement the code! 😆



Games

Most of the games are made with python and pygame!

The codes are easy to understand and variables are easy to understand.

So fix the code as you want!! 😆

Description of the games

Specific description of games