ERROR: ray::RolloutWorker.__init__()

[bertanimre]

Hello,
I would like to use or-gym supply chain environments for my project. I am trying to learn the environments now.

While following the "Using Ray and DFO to optimize a multi-echelon supply chain" tutorial, I faced an error called "-- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init()".

I didn't understand the reason for the error because I followed the steps without changing them.
Could you check it?

Thanks in advance.

Best regards,
Bertan

[hubbs5]

There was a breaking change made to Ray that we're working to address. What version are you running? Can you post the traceback?

[bertanimre]

Hello,
The Ray version is 1.10.0
I attached the screenshot of the problem.

Best regards,
Bertan

[wojnarabc]

Hello, facing similar issue with ray 2.0.0 while trying to run inv-management-quickstart.ipynb with InvManagement-v0/v1. Tutorial mentioned that 1.0.0 ray version should be used (runs fine there) but this one is already more than two years old. Is it something easily adjustable on our end ? Below my traceback:

`Exception has occurred: AssertionError
The actor died because of an error raised in its creation task, �[36mray::RolloutWorker.init()�[39m (pid=11044, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7fe7f1a01970>)
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in init
self._build_policy_map(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
self.policy_map.create_policy(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
self[policy_id] = create_policy_for_framework(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
return policy_class(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/algorithms/ppo/ppo_tf_policy.py", line 83, in init
base.init(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in init
) = self._init_action_fetches(timestep, explore)
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 627, in _init_action_fetches
) = self.exploration.get_exploration_action(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 84, in get_exploration_action
return self._get_tf_exploration_action_op(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 91, in get_tf_exploration_action_op
stochastic_actions = tf.cond(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 94, in
self.random_exploration.get_tf_exploration_action_op(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/random.py", line 138, in get_tf_exploration_action_op
action = tf.cond(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/random.py", line 67, in true_fn
action_dist.required_model_output_shape(
File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/models/tf/tf_action_dist.py", line 188, in required_model_output_shape
assert np.all(action_space.high == high)
AssertionError

During handling of the above exception, another exception occurred:

File "/Users/{user}/Projects/or-gym/examples/inv-management-quickstart.py", line 42, in
agent = PPOTrainer(env=env_name,`

[wojnarabc]

Started to work on my end when added framework= "torch" to rl_config and modified attribute path in both pol_loss and vf_loss adding ['learner_stats'] level on fourth place.
i.e. i['info']['learner']['default_policy']['learner_stats']['policy_loss'] .
Tested on Ray 2.1.0 version.

[deter3]

@wojnarabc your solution is working now .

[QJZJ12]

Started to work on my end when added framework= "torch" to rl_config and modified attribute path in both pol_loss and vf_loss adding ['learner_stats'] level on fourth place. i.e. i['info']['learner']['default_policy']['learner_stats']['policy_loss'] . Tested on Ray 2.1.0 version.

Can you share the code instance for rl_config and how the error solved for ray 2.1.0 version?