[Bug]: [SWE-bench] Faild to cd. ModuleNotFoundError #231

kevin-support-bot · 2025-01-24T03:54:53Z

@BIJOY-SUST, Is this issue specific to OpenHands version 0.18.0, as it's not the latest?

BIJOY-SUST · 2025-01-25T06:49:22Z

specific to OpenHands version 0.18.0

SmartManoj · 2025-01-25T06:50:41Z

Could you let me know why you're using that version?

BIJOY-SUST · 2025-01-25T07:11:03Z

I’ve been using the 0.18.0 version since its release. However, I recently switched to version 0.21.0 and encountered an issue.

Instance scikit-learn__scikit-learn-25500 - 2025-01-25 01:59:42,199 - ERROR - ----------                                        
Error in instance [scikit-learn__scikit-learn-25500]: Failed to cd to /workspace/scikit-learn__scikit-learn__1.3: **CmdOutputObs
ervation (source=None, exit code=-1, metadata={                                                                                 
  "exit_code": -1,                                                                                                              
  "pid": -1,                                                                                                                    
  "username": null,                                                                                                             
  "hostname": null,                                                                                                             
  "working_dir": null,                                                                                                          
  "py_interpreter_path": null,                                                                                                  
  "prefix": "[Below is the output of the previous command.]\n",                                                                 
  "suffix": "\n[Your command \"cd /workspace/scikit-learn__scikit-learn__1.3\" is NOT executed. The previous command is still ru
nning - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact 
with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send
 other commands to interact with the current process, or send keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous co
mmand before sending your new command.]"   
-------------------------------
-------------------------------
    raise EvalException(msg)                                                                                                    
evaluation.utils.shared.EvalException: Failed to cd to /workspace/scikit-learn__scikit-learn__1.3: **CmdOutputObservation (sourc
e=None, exit code=-1, metadata={

SmartManoj · 2025-01-25T07:17:56Z

Would you run ls /testbed and ls /workspace in that container? Here , the folder is copied.

BIJOY-SUST · 2025-01-25T19:24:59Z

I couldn't find the specific container id from the issue, and after running docker ps -a, there are many containers I found. So it is not feasible to check each of them. However, I found that this error occurred if I am running more than one worker. For a single worker, it works fine. I think this issue occurred if there are parallel workers.

Another note: For parallel workers there is another kind of issue i noticed-

================ DOCKER BUILD STARTED ================                                                                                               
Instance sympy__sympy-24152 - 2025-01-25 04:46:05,989 - ERROR - [runtime ee4b4d13-2e77-413b-92ba-2e2e51510a2d-7e0041aa3ba95507] Error: Instance openh
ands-runtime-ee4b4d13-2e77-413b-92ba-2e2e51510a2d-7e0041aa3ba95507 FAILED to start container!                                                        
                                                                                                                                                     
Instance sympy__sympy-24152 - 2025-01-25 04:46:05,990 - ERROR - [runtime ee4b4d13-2e77-413b-92ba-2e2e51510a2d-7e0041aa3ba95507] 500 Server Error for 
http+docker://localhost/v1.47/containers/5e287512107eb31936ba37073688916026b3a861c248dac4dad0ebe42f7dd63b/start: Internal Server Error ("driver faile
d programming external connectivity on endpoint openhands-runtime-ee4b4d13-2e77-413b-92ba-2e2e51510a2d-7e0041aa3ba95507 (c278fa626eea3ce95fa977b0e674
602297f8d5c4221b9aba88febf265d475c4c): failed to bind port 0.0.0.0:58226/tcp: Error starting userland proxy: listen tcp4 0.0.0.0:58226: bind: address
 already in use")                                                                                                                                    
Instance sympy__sympy-24152 - 2025-01-25 04:46:06,010 - ERROR - ----------                                                                           
Error in instance [sympy__sympy-24152]: 500 Server Error for http+docker://localhost/v1.47/containers/5e287512107eb31936ba37073688916026b3a861c248dac
4dad0ebe42f7dd63b/start: Internal Server Error ("driver failed programming external connectivity on endpoint openhands-runtime-ee4b4d13-2e77-413b-92b
a-2e2e51510a2d-7e0041aa3ba95507 (c278fa626eea3ce95fa977b0e674602297f8d5c4221b9aba88febf265d475c4c): failed to bind port 0.0.0.0:58226/tcp: Error star
ting userland proxy: listen tcp4 0.0.0.0:58226: bind: address already in use"). Stacktrace:                                                          
Traceback (most recent call last):                                                                                                                   
  File "/home/user/.cache/pypoetry/virtualenvs/openhands-ai-lecMOyrf-py3.12/lib/python3.12/site-packages/docker/api/client.py", line 275, in _raise
_for_status                                                                                                                                          
    response.raise_for_status()                                                                                                                      
  File "/home/user/.cache/pypoetry/virtualenvs/openhands-ai-lecMOyrf-py3.12/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_f
or_status                                                                                                                                            
    raise HTTPError(http_error_msg, response=self)                                                                                                   
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.47/containers/5e287512107eb31936ba37073688
916026b3a861c248dac4dad0ebe42f7dd63b/start                                                                                                           
                                                                                                                                                     
The above exception was the direct cause of the following exception:

I changed the sandbox_config.py according to the comment mentioned in a different issue thread. The following is the current version-

    # remote_runtime_api_url: str = Field(default='http://localhost:8000')
    remote_runtime_api_url: str | None = Field(default=None)

SmartManoj · 2025-01-26T05:23:36Z

First issue: another container may use the same port and already started and it checks the folder in another container.

Root cause: when running multiple workers, same port is being used.
Here, give unique port for each instance using a dictionary.

BIJOY-SUST · 2025-01-26T20:42:30Z

I will try to provide custom ports in the code section you mentioned. Thank you.

Apart from this issue, I am facing another issue-
I am trying to do the inference on swe-bench-lite and experienced "too many open files" error-

Using openhands 0.21.0 version

raise ConnectionError(err, request=request)                                                                                         [45/1959]
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(24, 'Too many open files'))                                                 
Exception ignored in atexit callback: <bound method DockerRuntime.close of <openhands.runtime.impl.docker.docker_runtime.DockerRuntime object at 
0x7ff4c5edaff0>>                                                                                                                                 
Traceback (most recent call last):                                                                                                               
  File "project/openhands/runtime/impl/docker/docker_runtime.py", line 371, in close                       
    stop_all_containers(close_prefix)                                                                                                            
  File "project/openhands/runtime/impl/docker/containers.py", line 7, in stop_all_containers               
    containers = docker_client.containers.list(all=True)                                                                                         
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                         
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/models/containers.py", line 101
8, in list
    containers.append(self.get(r['Id']))
                      ^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/models/containers.py", line 954
, in get
    resp = self.client.api.inspect_container(container_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/utils/decorators.py", line 19, 
in wrapped
    return f(self, resource_id, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/api/container.py", line 794, in
 inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/utils/decorators.py", line 44, 
in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/docker/api/client.py", line 246, in _g
et
    return self.get(url, **self._set_request_timeout(kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 602, in ge
t
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 589, in re
quest
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/requests/sessions.py", line 703, in se
nd
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.cache/pypoetry/virtualenvs/openhands-ai-xxSbwZmD-py3.12/lib/python3.12/site-packages/requests/adapters.py", line 682, in se
nd
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(24, 'Too many open files'))
Exception ignored in atexit callback: <function stop_all_runtime_containers at 0x7ff5763272e0>
Traceback (most recent call last):

SmartManoj · 2025-01-27T01:29:32Z

https://stackoverflow.com/a/35236738/5033247

BIJOY-SUST · 2025-01-31T03:49:28Z

I’ve been using the 0.18.0 version since its release. However, I recently switched to version 0.21.0 and encountered an issue.

Instance scikit-learn__scikit-learn-25500 - 2025-01-25 01:59:42,199 - ERROR - ----------                                        
Error in instance [scikit-learn__scikit-learn-25500]: Failed to cd to /workspace/scikit-learn__scikit-learn__1.3: **CmdOutputObs
ervation (source=None, exit code=-1, metadata={                                                                                 
  "exit_code": -1,                                                                                                              
  "pid": -1,                                                                                                                    
  "username": null,                                                                                                             
  "hostname": null,                                                                                                             
  "working_dir": null,                                                                                                          
  "py_interpreter_path": null,                                                                                                  
  "prefix": "[Below is the output of the previous command.]\n",                                                                 
  "suffix": "\n[Your command \"cd /workspace/scikit-learn__scikit-learn__1.3\" is NOT executed. The previous command is still ru
nning - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact 
with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send
 other commands to interact with the current process, or send keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous co
mmand before sending your new command.]"   
-------------------------------
-------------------------------
    raise EvalException(msg)                                                                                                    
evaluation.utils.shared.EvalException: Failed to cd to /workspace/scikit-learn__scikit-learn__1.3: **CmdOutputObservation (sourc
e=None, exit code=-1, metadata={

Thanks for your earlier response. Could you please provide more details on how to address this Failed to cd to issue? The Failed to cd to error is occurring quite frequently.

SmartManoj · 2025-01-31T03:55:30Z

First issue: another container may use the same port and already started and it checks the folder in another container.

It's due to the port conflict.
Let's have two instances A and B.
If both uses the same port P, and if container B started before container A,
then the program A will check in container B and cd failed error will come.
Then later, if container A started, failed to bind error will occur.

Resolution for both issues: Unique port for each instance.

SmartManoj · 2025-01-31T04:04:41Z

Snippet to map each port.

from datasets import load_dataset
dataset = load_dataset(
                'princeton-nlp/SWE-bench_Lite',
                cache_dir='./cache',
                verification_mode='no_checks',
                num_proc=4,
                split='test',
            )

port_range = 63000
port_mapping = {}
for i in range(len(dataset)):
    port_mapping[dataset[i]['instance_id']] = port_range + i
print(port_mapping)

SmartManoj · 2025-01-31T04:06:52Z

Did you check about swebench_verified_mini?

Fix #231

SmartManoj · 2025-01-31T04:33:04Z

Would you apply this commit and check if the issue resolves for you?

BIJOY-SUST · 2025-01-31T04:36:19Z

I’m applying this commit right now. I’ll keep you updated if the issue is resolved.

BIJOY-SUST · 2025-01-31T17:28:52Z

I applied the commit, but it didn’t resolve the “failed to cd” issue. Currently, I have two issues:

“failed to cd”: I encountered the same issue after applying the mentioned commit.

"exit_code": -1,                                                                                                                              
"pid": -1,                                                                                                                                    
"username": null,                                                                                                                             
"hostname": null,                                                                                                                             
"working_dir": null,                                                                                                                          
"py_interpreter_path": null,                                                                                                                  
"prefix": "[Below is the output of the previous command.]\n",                                                                                 
"suffix": "\n[Your command \"cd /workspace/django__django__3.2\" is NOT executed. The previous command is still running - You CANNOT send new 
commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longe
r to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or sen
d keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous command before sending your new command.]"                                    
})**                                                                                                                                            
--BEGIN AGENT OBSERVATION--                                                                                                                     
[Below is the output of the previous command.]                                                                                                  
                                                                                                                                              
[Your command "cd /workspace/django__django__3.2" is NOT executed. The previous command is still running - You CANNOT send new commands until th
e previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see addition
al output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys ("C-c", "C
-z", "C-d") to interrupt/kill the previous command before sending your new command.]                                                            
--END AGENT OBSERVATION--. Stacktrace:                                                                                                          
Traceback (most recent call last):                                                                                                              
File "/project/evaluation/utils/shared.py", line 309, in _process_instance_wrapper    
  result = process_instance_func(instance, metadata, use_mp, **kwargs)                                                                        
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                        
File "/project/evaluation/benchmarks/swe_bench/run_infer.py", line 443, in process_ins
tance                                                                                                                                           
  return_val = complete_runtime(runtime, instance)                                                                                            
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                            
File "/project/evaluation/benchmarks/swe_bench/run_infer.py", line 332, in complete_ru
ntime                                                                                                                                           
  assert_and_raise(                                                                                                                           
File "/project/evaluation/utils/shared.py", line 286, in assert_and_raise             
  raise EvalException(msg)                                                                                                                    
evaluation.utils.shared.EvalException: Failed to cd to /workspace/django__django__3.2: **CmdOutputObservation (source=None, exit code=-1, metada
ta={                                                                                                                                            
"exit_code": -1,                                                                                                                              
"pid": -1,                                                                                                                                    
"username": null,                                                                                                                             
"hostname": null,                                                                                                                             
"working_dir": null,                                                                                                                          
"py_interpreter_path": null,                                                                                                                  
"prefix": "[Below is the output of the previous command.]\n",                                                                                 
"suffix": "\n[Your command \"cd /workspace/django__django__3.2\" is NOT executed. The previous command is still running - You CANNOT send new 
commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longe
r to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or sen
d keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous command before sending your new command.]"                                    
})**                                                                                                                                            
--BEGIN AGENT OBSERVATION--
[Below is the output of the previous command.]

I also encountered the following error: “container not found.” For more details, please refer to this link for the inference logs for this instance.

self._raise_for_status(response)
File "/project/.cache/pypoetry/virtualenvs/openhands-ai-PxnfiNA9-py3.12/lib/python3.12/site-packages/docker/api/client.py", line 277, in _raise_for_status
  raise create_api_error_from_http_exception(e) from e
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/.cache/pypoetry/virtualenvs/openhands-ai-PxnfiNA9-py3.12/lib/python3.12/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
  raise cls(e, response=response, explanation=explanation) from e
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.47/containers/openhands-runtime-scikit-learn__scikit-learn-15512/json: Not Found ("No such container: openhands-runtime-scikit-learn__scikit-learn-15512")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/project/evaluation/utils/shared.py", line 309, in _process_instance_wrapper
  result = process_instance_func(instance, metadata, use_mp, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/evaluation/benchmarks/swe_bench/run_infer.py", line 418, in process_instance
  call_async_from_sync(runtime.connect)
File "/project/openhands/utils/async_utils.py", line 50, in call_async_from_sync
  result = future.result()
           ^^^^^^^^^^^^^^^
File "/project/.conda/envs/openhands_latest/lib/python3.12/concurrent/futures/_base.py", line 449, in result
  return self.__get_result()
         ^^^^^^^^^^^^^^^^^^^
File "/project/.conda/envs/openhands_latest/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
  raise self._exception
File "/project/.conda/envs/openhands_latest/lib/python3.12/concurrent/futures/thread.py", line 58, in run
  result = self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/openhands/utils/async_utils.py", line 44, in run
  return asyncio.run(arun())
         ^^^^^^^^^^^^^^^^^^^
File "/project/.conda/envs/openhands_latest/lib/python3.12/asyncio/runners.py", line 194, in run
  return runner.run(main)
         ^^^^^^^^^^^^^^^^
File "/project/.conda/envs/openhands_latest/lib/python3.12/asyncio/runners.py", line 118, in run
  return self._loop.run_until_complete(task)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/.conda/envs/openhands_latest/lib/python3.12/asyncio/base_events.py", line 664, in run_until_complete
  return future.result()
         ^^^^^^^^^^^^^^^
File "/project/openhands/utils/async_utils.py", line 37, in arun
  result = await coro
           ^^^^^^^^^^
File "/project/openhands/runtime/impl/docker/docker_runtime.py", line 135, in connect
  self.runtime_container_image = build_runtime_image(
                                 ^^^^^^^^^^^^^^^^^^^^
File "/project/openhands/runtime/utils/runtime_build.py", line 137, in build_runtime_image
  result = build_runtime_image_in_folder(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/openhands/runtime/utils/runtime_build.py", line 232, in build_runtime_image_in_folder
  _build_sandbox_image(
File "/project/openhands/runtime/utils/runtime_build.py", line 361, in _build_sandbox_image
  image_name = runtime_builder.build(
               ^^^^^^^^^^^^^^^^^^^^^^
File "/project/openhands/runtime/builder/docker.py", line 163, in build
  raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['docker', 'buildx', 'build', '--progress=plain', '--build-arg=OPENHANDS_RUNTIME_VERSION=0.21.0', '--build-arg=OPENHANDS_RUNTIME_BUILD_TIME=2025-01-31T14:54:32.083523', '--tag=ghcr.io/all-hands-ai/runtime:oh_v0.21.0_d8d8j73ho5y441j5_g7m4yn2837jngz2k', '--load', '--platform=linux/amd64', '/tmp/tmp4vda6inc']' returned non-zero exit status 1.

----------[The above error occurred. Retrying... (attempt 2 of 5)]----------

SmartManoj · 2025-02-01T03:11:57Z

openhands-runtime-scikit-learn__scikit-learn-15512

Now this is the new container name. Would you check the container logs?

Would you apply this commit to see why the buildx command failed?

BIJOY-SUST · 2025-02-01T05:15:50Z

Do you have further details on how to address this Failed to cd to issue?

Now this is the new container name. Would you check the container logs?

-> There is no such container in this name in the container list.

I believe the buildx command failed because it couldn’t locate the container, as mentioned in the inference logs. Interestingly, when I rerun the inference process for that specific instance, it completed without any issues. However, the same issue persisted for different instances later on.

Would you apply this commit to see why the buildx command failed?

I applied this commit and reran the inference process.

SmartManoj · 2025-02-01T05:19:07Z

Now this is the new container name. Would you check the container logs?

Now this is the new container name format. Would you check the container logs for the django?

BIJOY-SUST · 2025-02-02T18:07:12Z

There are two issues now and these issues occurred on multiple instances-

Failed to cd

File "/project/evaluation/utils/shared.py", line 286, in assert_and_raise
raise EvalException(msg)
evaluation.utils.shared.EvalException: Failed to cd to /workspace/django__django__3.2: **CmdOutputObservation (source=None, exit code=-1, metada
ta={

Not Found ("No such container...

docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.47/containers/openhands-runtime-scikit-learn__scikit-learn-14983/json: Not Found ("No such container: openhands-runtime-scikit-learn__scikit-learn-14983")

Running openhands again and will share the logs with you.

SmartManoj · 2025-02-03T02:36:49Z

Would you run only these two instances simulataneously?

BIJOY-SUST · 2025-02-03T16:27:46Z

Would you run only these two instances simulataneously?

In this time, they ran without any problems. However, the issue of “failed to cd” still persists.

After applying the mapping port commit you provided, it generates only empty patches. This commit

The normal steps begin at 1 and then 2, 3, 4, and so on, up to 30. However, for this commit, it starts randomly, such as 13, performs 0 to 5(n) steps, and then generates empty patches.

SmartManoj · 2025-02-03T16:32:16Z

Is the llm context_window small? Which model are you using?

BIJOY-SUST · 2025-02-03T16:34:19Z

128k
I am using various kinds of model. This issues for this commit persists in all models. There is not related to model I think.

BIJOY-SUST · 2025-02-03T16:35:37Z

For temperature 0, I checked using this commit and without it. Without it, there a non-empty patch generated, but when I use this commit, there’s an empty patch.

SmartManoj · 2025-02-03T16:36:47Z

Even when running a single instance, did it produce an empty patch?

BIJOY-SUST · 2025-02-03T16:38:46Z

Yes, I checked running a single instance and it produce an empty patch. Is this commit worked fine from your side?

SmartManoj · 2025-02-03T16:41:09Z

Would you provide the console logs?

SmartManoj · 2025-02-03T16:44:57Z

Did you check the remote runtime?

BIJOY-SUST · 2025-02-03T16:53:15Z

Did you check the remote runtime?

No.

After applying this commit. Can you please just verify this commit and update it if necessary. Because after applying, it produce empty patches. Specially for the following line of codes from your commit I think (Maybe I am wrong here)-

if os.environ.get('RUNTIME') != 'remote':
        runtime = create_runtime(config, sid=instance.instance_id)
else:
        runtime = create_runtime(config)

Would you provide the console logs?

You can see from the below logs, it start at step 13 randomly I think.

--BEGIN AGENT OBSERVATION--
/opt/miniconda3/envs/testbed/bin/python
[The command completed with exit code 0.]
[Current working directory: /workspace/astropy__astropy__4.3]
[Python interpreter: /opt/miniconda3/envs/testbed/bin/python]
[Command finished with exit code 0] 
--END AGENT OBSERVATION--
21:33:20 - openhands:INFO: run_infer.py:309 - ------------------------------
21:33:20 - openhands:INFO: run_infer.py:310 - END Runtime Initialization Fn
21:33:20 - openhands:INFO: run_infer.py:311 - ------------------------------
21:33:20 - openhands:INFO: run_infer.py:65 - Length of the issue statement: 1246
21:33:20 - openhands:INFO: codeact_agent.py:94 - Function calling not enabled for model /llm_model/. Mocking function calling via prompting.
21:33:20 - openhands:INFO: base.py:253 - [runtime astropy__astropy-12907] Selected repo: None, loading microagents from /workspace/.openhands
/microagents (inside runtime)
21:33:20 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.LOADING to 
AgentState.RUNNING
[Agent Controller default] LEVEL 0 LOCAL STEP 13 GLOBAL STEP 13
21:33:20 - openhands:WARNING: stuck.py:307 - Action, Observation pattern detected
21:33:20 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.RUNNING to 
AgentState.ERROR
21:33:20 - openhands:ERROR: loop.py:22 - AgentStuckInLoopError: Agent got stuck in a loop
21:33:20 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.ERROR to Ag
entState.ERROR
21:33:21 - openhands:INFO: run_infer.py:324 - ------------------------------
21:33:21 - openhands:INFO: run_infer.py:325 - BEGIN Runtime Completion Fn
21:33:21 - openhands:INFO: run_infer.py:326 - ------------------------------
21:33:21 - ACTION
**CmdRunAction (source=None, is_input=False)**
COMMAND:
cd /workspace/astropy__astropy__4.3 
21:33:22 - OBSERVATION
**CmdOutputObservation (source=None, exit code=0, metadata={
  "exit_code": 0,
  "pid": -1,
  "username": "root",
  "hostname": "fb0b87b000c8",
  "working_dir": "/workspace/astropy__astropy__4.3",
  "py_interpreter_path": "/opt/miniconda3/envs/testbed/bin/python",
  "prefix": "",
  "suffix": "\n[The command completed with exit code 0.]"
})**
--BEGIN AGENT OBSERVATION--

[The command completed with exit code 0.]
[Current working directory: /workspace/astropy__astropy__4.3]
[Python interpreter: /opt/miniconda3/envs/testbed/bin/python]

SmartManoj · 2025-02-03T17:05:36Z

Do you want to create the container for each run?
If so,
You can pass instance_id via env and remove the sid parama

Else,
Seems the previous state is restored. You can override the behavior here. (Will link the upstream location)

Kevin/openhands/core/setup.py

Lines 95 to 96 in 9fa0e6b

    
           if config.dont_restore_state: 
        
               initial_state = None

SmartManoj · 2025-02-03T17:08:30Z

Are you using the seed parameter?

BIJOY-SUST · 2025-02-03T17:19:52Z

Are you using the seed parameter?

No

Do you want to create the container for each run?
If so,
You can pass instance_id via env and remove the sid parama

Can you please provide the code Snippet for this? Thanks!!!

Else,
Seems the previous state is restored. You can override the behavior here. (Will link the upstream location)

 if config.dont_restore_state: 
     initial_state = None

You are referring these two lines in setup.py. Am I correct?

SmartManoj · 2025-02-03T17:28:54Z

Would you check this commit ? (not tested yet)

You are referring these two lines in setup.py. Am I correct?

yes.

BIJOY-SUST · 2025-02-03T18:32:08Z

I chose the second option you suggested and edited the setup.py. Initial_state = None. But it didn't work. Randomly start at step 0. Without applying the commits for mapping port, it worked fine.

13:05:18 - openhands:INFO: run_infer.py:310 - END Runtime Initialization Fn                                                                  
13:05:18 - openhands:INFO: run_infer.py:311 - ------------------------------                                                                 
13:05:18 - openhands:INFO: run_infer.py:65 - Length of the issue statement: 1871                                                             
13:05:18 - openhands:INFO: codeact_agent.py:94 - Function calling not enabled for model /llm_model/. Mocking function calling via prompting.                                                                           
13:05:18 - openhands:INFO: base.py:253 - [runtime astropy__astropy-14182] Selected repo: None, loading microagents from /workspace/.openhands
/microagents (inside runtime)                                                                                                                
13:05:18 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.LOADING to 
AgentState.RUNNING                                                                                                                           
[Agent Controller default] LEVEL 0 LOCAL STEP 0 GLOBAL STEP 0                                                                                
13:05:18 - openhands:WARNING: stuck.py:265 - Repeated MessageAction with source=AGENT detected                                               
13:05:18 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.RUNNING to 
AgentState.ERROR                                                                                                                             
13:05:18 - openhands:ERROR: loop.py:22 - AgentStuckInLoopError: Agent got stuck in a loop                                                    
13:05:18 - openhands:INFO: agent_controller.py:447 - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.ERROR to Ag
entState.ERROR                                                                                                                               
13:05:19 - openhands:INFO: run_infer.py:324 - ------------------------------                                                                 
13:05:19 - openhands:INFO: run_infer.py:325 - BEGIN Runtime Completion Fn                                                                    
13:05:19 - openhands:INFO: run_infer.py:326 - ------------------------------                                                                 
13:05:19 - ACTION                                                                                                                            
**CmdRunAction (source=None, is_input=False)**                                                                                               
COMMAND:                                                                                                                                     
cd /workspace/astropy__astropy__5.1

Would you check this commit ? (not tested yet)

Checked but it didn't work. Same result as above.

--

Else,
Seems the previous state is restored. You can override the behavior here. (Will link the upstream location)

Could please check how we can achieve this? Mapping port for instances is great idea and hopeful that it will solve failed to cd issue. If we can achieve this, this would be great. Thanks.

BIJOY-SUST · 2025-02-04T02:04:21Z

One thing I noticed- I didn't use the port mapping, just ran the openhands using 4 workers and faced failed to cd issue for an instance, and code execution stopped after 5 attempts. Then I stopped and removed all the containers and removed all the images also. Cleared the docker cache. THEN RAN openhands for that particular instance using worker 1 and faced failed to cd issue again.

..........................
2025-02-03 18:02:59,076 - INFO - [Agent Controller default] LEVEL 0 LOCAL STEP 12 GLOBAL STEP 12
2025-02-03 18:03:02,313 - INFO - [Agent Controller default] LEVEL 0 LOCAL STEP 13 GLOBAL STEP 13
2025-02-03 18:03:35,862 - INFO - [Agent Controller default] LEVEL 0 LOCAL STEP 14 GLOBAL STEP 14
2025-02-03 18:03:40,236 - INFO - [Agent Controller default] LEVEL 0 LOCAL STEP 15 GLOBAL STEP 15
2025-02-03 18:03:40,236 - WARNING - Action, Observation pattern detected
2025-02-03 18:03:40,236 - INFO - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.ERROR
2025-02-03 18:03:40,237 - ERROR - AgentStuckInLoopError: Agent got stuck in a loop
2025-02-03 18:03:40,238 - INFO - [Agent Controller default] Setting agent(CodeActAgent) state from AgentState.ERROR to AgentState.ERROR
2025-02-03 18:03:40,510 - INFO - ------------------------------
2025-02-03 18:03:40,510 - INFO - BEGIN Runtime Completion Fn
2025-02-03 18:03:40,510 - INFO - ------------------------------
2025-02-03 18:03:40,511 - INFO - **CmdRunAction (source=None, is_input=False)**
COMMAND:
cd /workspace/sphinx-doc__sphinx__7.1
2025-02-03 18:03:40,519 - INFO - **CmdOutputObservation (source=None, exit code=-1, metadata={
  "exit_code": -1,
  "pid": -1,
  "username": null,
  "hostname": null,
  "working_dir": null,
  "py_interpreter_path": null,
  "prefix": "[Below is the output of the previous command.]\n",
  "suffix": "\n[Your command \"cd /workspace/sphinx-doc__sphinx__7.1\" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous command before sending your new command.]"
})**
--BEGIN AGENT OBSERVATION--
[Below is the output of the previous command.]

[Your command "cd /workspace/sphinx-doc__sphinx__7.1" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command.]
--END AGENT OBSERVATION--
2025-02-03 18:03:41,844 - ERROR - ----------
Error in instance [sphinx-doc__sphinx-11445]: Failed to cd to /workspace/sphinx-doc__sphinx__7.1: **CmdOutputObservation (source=None, exit code=-1, metadata={
  "exit_code": -1,
  "pid": -1,
  "username": null,
  "hostname": null,
  "working_dir": null,
  "py_interpreter_path": null,
  "prefix": "[Below is the output of the previous command.]\n",
  "suffix": "\n[Your command \"cd /workspace/sphinx-doc__sphinx__7.1\" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous command before sending your new command.]"
})**
--BEGIN AGENT OBSERVATION--
[Below is the output of the previous command.]

[Your command "cd /workspace/sphinx-doc__sphinx__7.1" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command.]
--END AGENT OBSERVATION--. Stacktrace:
Traceback (most recent call last):
  File "/openhands/evaluation/utils/shared.py", line 309, in _process_instance_wrapper
    result = process_instance_func(instance, metadata, use_mp, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openhands/evaluation/benchmarks/swe_bench/run_infer.py", line 444, in process_instance
    return_val = complete_runtime(runtime, instance)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openhands/evaluation/benchmarks/swe_bench/run_infer.py", line 333, in complete_runtime
    assert_and_raise(
  File "/openhands/evaluation/utils/shared.py", line 286, in assert_and_raise
    raise EvalException(msg)
evaluation.utils.shared.EvalException: Failed to cd to /workspace/sphinx-doc__sphinx__7.1: **CmdOutputObservation (source=None, exit code=-1, metadata={
  "exit_code": -1,
  "pid": -1,
  "username": null,
  "hostname": null,
  "working_dir": null,
  "py_interpreter_path": null,
  "prefix": "[Below is the output of the previous command.]\n",
  "suffix": "\n[Your command \"cd /workspace/sphinx-doc__sphinx__7.1\" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys (\"C-c\", \"C-z\", \"C-d\") to interrupt/kill the previous command before sending your new command.]"
})**
--BEGIN AGENT OBSERVATION--
[Below is the output of the previous command.]

[Your command "cd /workspace/sphinx-doc__sphinx__7.1" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output of the previous command by sending empty command '', send other commands to interact with the current process, or send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command.]
--END AGENT OBSERVATION--

----------[The above error occurred. Retrying... (attempt 1 of 5)]----------

.............

SmartManoj · 2025-02-04T03:33:59Z

Instance scikit-learn__scikit-learn-25500 - 2025-01-25 01:59:42,199 - ERROR - ----------
Error in instance [scikit-learn__scikit-learn-25500]: Failed to cd to /workspace/scikit-learn__scikit-learn__1.3: **CmdOutputObs
ervation (source=None, exit code=-1, metadata={

In the last week's error itself, it was failed due to the previous command running. Would you apply this commit only and check?

Related discussion #118

SmartManoj added a commit that referenced this issue Jan 31, 2025

Ensure unique port for multiple workers

9887330

Fix #231

[Bug]: [SWE-bench] Faild to cd. ModuleNotFoundError #231

[Bug]: [SWE-bench] Faild to cd. ModuleNotFoundError #231

Comments

kevin-support-bot bot commented Jan 24, 2025

BIJOY-SUST commented Jan 25, 2025

SmartManoj commented Jan 25, 2025

BIJOY-SUST commented Jan 25, 2025 • edited Loading

SmartManoj commented Jan 25, 2025 • edited Loading

BIJOY-SUST commented Jan 25, 2025 • edited Loading

SmartManoj commented Jan 26, 2025 • edited Loading

BIJOY-SUST commented Jan 26, 2025 • edited Loading

SmartManoj commented Jan 27, 2025

BIJOY-SUST commented Jan 31, 2025 • edited Loading

SmartManoj commented Jan 31, 2025

SmartManoj commented Jan 31, 2025 • edited Loading

SmartManoj commented Jan 31, 2025

SmartManoj commented Jan 31, 2025

BIJOY-SUST commented Jan 31, 2025

BIJOY-SUST commented Jan 31, 2025 • edited Loading

SmartManoj commented Feb 1, 2025 • edited Loading

BIJOY-SUST commented Feb 1, 2025

SmartManoj commented Feb 1, 2025 • edited Loading

BIJOY-SUST commented Feb 2, 2025

SmartManoj commented Feb 3, 2025

BIJOY-SUST commented Feb 3, 2025 • edited Loading

SmartManoj commented Feb 3, 2025 • edited Loading

BIJOY-SUST commented Feb 3, 2025 • edited Loading

BIJOY-SUST commented Feb 3, 2025 • edited Loading

SmartManoj commented Feb 3, 2025

BIJOY-SUST commented Feb 3, 2025

SmartManoj commented Feb 3, 2025

SmartManoj commented Feb 3, 2025

BIJOY-SUST commented Feb 3, 2025 • edited Loading

SmartManoj commented Feb 3, 2025 • edited Loading

SmartManoj commented Feb 3, 2025

BIJOY-SUST commented Feb 3, 2025 • edited Loading

SmartManoj commented Feb 3, 2025

BIJOY-SUST commented Feb 3, 2025 • edited Loading

BIJOY-SUST commented Feb 4, 2025 • edited Loading

SmartManoj commented Feb 4, 2025 • edited Loading

BIJOY-SUST commented Jan 25, 2025 •

edited

Loading

SmartManoj commented Jan 25, 2025 •

edited

Loading

BIJOY-SUST commented Jan 25, 2025 •

edited

Loading

SmartManoj commented Jan 26, 2025 •

edited

Loading

BIJOY-SUST commented Jan 26, 2025 •

edited

Loading

BIJOY-SUST commented Jan 31, 2025 •

edited

Loading

SmartManoj commented Jan 31, 2025 •

edited

Loading

BIJOY-SUST commented Jan 31, 2025 •

edited

Loading

SmartManoj commented Feb 1, 2025 •

edited

Loading

SmartManoj commented Feb 1, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

SmartManoj commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

SmartManoj commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 3, 2025 •

edited

Loading

BIJOY-SUST commented Feb 4, 2025 •

edited

Loading

SmartManoj commented Feb 4, 2025 •

edited

Loading