ShippingEnv

A from-scratch reinforcement-learning environment: haul cargo between ports on a world map.

Reward 0.00Fuel 200.0Bound for port 3

Steer with the arrows. Each move is scored by a port of the environment's step on its real map, starting at port 0 with no cargo aboard, so loading and losing cargo are left out.

On the environment's 100×100 map, each move scores +2 if it brings the ship closer to its destination and −2 if not, plus −1 for moving over water or −5 for running into land, where the ship stays put. That dense signal steers the ship long before the +10 for arriving and +2 per unit of cargo still aboard.

A ship starts at a port with a destination, loads cargo and fuel, and navigates a 100×100 map of the North Atlantic without running aground or out of fuel, then gets a new destination on arrival. The repo ships the environment plus three interchangeable learners (deep Q-network, tabular SARSA, Monte Carlo tree search) behind one CLI.

Why build the environment too

Routing, loading and refueling is a real sequential-decision problem: a wrong load early strands you later. Writing the environment instead of taking one off the shelf meant owning the dynamics (noisy fuel burn, cargo that can wash overboard) and the reward shaping, then comparing a value-based learner, an on-policy tabular one, and a planner on exactly the same task.

How it's put together

main.py builds the environment and hands it to the agent you pick; DQN, SARSA and MCTS sit behind one BaseAgent interface and run the same loop of step(action) in, state and reward out. MCTS learns nothing between moves: it plays 50 simulations on fresh copies of the environment and picks the best first move.

Trade-offs

Flat action head with masking. Lets a standard DQN handle a parameterized action space. The downside: A large, sparse output layer (200 of the 259 outputs are fuel amounts).

Discretized state for SARSA. Bucketing the state (fuel in 20-unit buckets) keeps the Q-table tractable. The downside: Coarse decisions near the edges of each bucket.

Next: Order forecasting for Papa JohnsPer-store, per-hour order forecasts set against what the staff on shift can handle.

ShippingEnv

A from-scratch reinforcement-learning environment: haul cargo between ports on a world map.

Reward 0.00Fuel 200.0Bound for port 3

Steer with the arrows. Each move is scored by a port of the environment's step on its real map, starting at port 0 with no cargo aboard, so loading and losing cargo are left out.

Why build the environment too

How it's put together

Trade-offs

Flat action head with masking. Lets a standard DQN handle a parameterized action space. The downside: A large, sparse output layer (200 of the 259 outputs are fuel amounts).

Discretized state for SARSA. Bucketing the state (fuel in 20-unit buckets) keeps the Q-table tractable. The downside: Coarse decisions near the edges of each bucket.

Next: Order forecasting for Papa JohnsPer-store, per-hour order forecasts set against what the staff on shift can handle.