time.sleep(0) or greenthread.sleep(0) is docuemnted as the canonical way of expressing a cooperative yield

In #932 asserts were converted in to a runtime error
while that may be correct in some cases the docs string of greenthread.sleep states
that calling sleep(0) is the canonical way of expressing a cooperative yield

eventlet/eventlet/greenthread.py

Lines 25 to 35 in 3450e6a

    
               *seconds* may be specified as an integer, or a float if fractional seconds 
        
               are desired. Calling :func:`~greenthread.sleep` with *seconds* of 0 is the 
        
               canonical way of expressing a cooperative yield. For example, if one is 
        
               looping over a large list performing an expensive calculation without 
        
               calling any socket methods, it's a good idea to call ``sleep(0)`` 
        
               occasionally; otherwise nothing else will run. 
        
               """ 
        
               hub = hubs.get_hub() 
        
               current = getcurrent() 
        
               if hub.greenlet is current: 
        
                   raise RuntimeError('do not call blocking functions from the mainloop')

nova and opentack in general has also codeifed this pattern in our contributor docs
https://docs.openstack.org/nova/latest/reference/threading.html#yielding-the-thread-in-long-running-tasks

as a result calling time.sleep(0) or greenthread.sleep(0) is quite common in OpenStack code.

as of PR #932 the previsly silent assertion is now a runtime error.

this manifests in trackbacks like this

https://paste.opendev.org/show/bGPgfURx1cZYOsgmtDyw/

as reported in https://bugs.launchpad.net/watcher/+bug/2086710

in that traceback we see oslo.db define a _thread_yeild function

https://github.com/openstack/oslo.db/blob/21c484ffb26ed03cbb1ca9dd7fd27fb19a1d4b36/oslo_db/sqlalchemy/engines.py#L43-L52

which calls time.sleep(0)

this is registered as a callback handeler for the sqlalchemy checkin event

https://github.com/openstack/oslo.db/blob/21c484ffb26ed03cbb1ca9dd7fd27fb19a1d4b36/oslo_db/sqlalchemy/engines.py#L361

which is emmited by sqlalchemy _finalize_fairy method

https://github.com/sqlalchemy/sqlalchemy/blob/214088c42ad61794e315174c41ee92a3c408ae6c/lib/sqlalchemy/pool/base.py#L1034-L1035

I belive the correct behavior in evently would be to only raise a runtime error if and only if seconds is not 0 and the current thread is the main thread.

I will take this on in a bit.

by the way the full set of logs also have additional exceptions

https://paste.opendev.org/show/bPfXhgQUM7Nu2YZZilAq/

so i think this other assertion/exption can also be raised with sqlachmeys current usage parterns

https://github.com/eventlet/eventlet/pull/932/files#diff-959fa03cb091386d7ad8eb6740341037ea2c345b30681ddd29f2b9106f7621d5R139

i am unsure if that is correct or not in that case.
i might just be misreading that trackback by the way, im not sure if the excption raised form the trampoline funciton in that case is the same excption that form the time.sleep(0) execption beign reraied.

@SeanMooney

Thank you @SeanMooney

Thanks, that resolves the first of the two exceptions that i am seeing

the other one is

 [-] Exception during reset or similar: RuntimeError: do not call blocking functions from the mainloop                  
 Traceback (most recent call last):                                                                                     
   File "/opt/stack/data/venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 986, in _finalize_fairy       
     fairy._reset(                                                                                                      
   File "/opt/stack/data/venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1432, in _reset               
     pool._dialect.do_rollback(self)                                                                                    
   File "/opt/stack/data/venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 699, in do_rollback      
     dbapi_connection.rollback()                                                                                        
   File "/opt/stack/data/venv/lib/python3.12/site-packages/pymysql/connections.py", line 493, in rollback               
     self._read_ok_packet()                                                                                             
   File "/opt/stack/data/venv/lib/python3.12/site-packages/pymysql/connections.py", line 453, in _read_ok_packet        
     pkt = self._read_packet()                                                                                          
           ^^^^^^^^^^^^^^^^^^^                                                                                          
   File "/opt/stack/data/venv/lib/python3.12/site-packages/pymysql/connections.py", line 744, in _read_packet           
     packet_header = self._read_bytes(4)                                                                                
                     ^^^^^^^^^^^^^^^^^^^                                                                                
   File "/opt/stack/data/venv/lib/python3.12/site-packages/pymysql/connections.py", line 782, in _read_bytes            
     data = self._rfile.read(num_bytes)                                                                                 
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                 
   File "/usr/lib/python3.12/socket.py", line 707, in readinto                                                          
     return self._sock.recv_into(b)                                                                                     
            ^^^^^^^^^^^^^^^^^^^^^^^                                                                                     
   File "/home/ubuntu/repos/eventlet/eventlet/greenio/base.py", line 358, in recv_into                                  
     return self._recv_loop(self.fd.recv_into, 0, buffer, nbytes, flags)                                                
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                
   File "/home/ubuntu/repos/eventlet/eventlet/greenio/base.py", line 346, in _recv_loop                                 
     self._read_trampoline()                                                                                            
   File "/home/ubuntu/repos/eventlet/eventlet/greenio/base.py", line 314, in _read_trampoline                           
     self._trampoline(                                                                                                  
   File "/home/ubuntu/repos/eventlet/eventlet/greenio/base.py", line 206, in _trampoline                                
     return trampoline(fd, read=read, write=write, timeout=timeout,                                                     
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                     
   File "/home/ubuntu/repos/eventlet/eventlet/hubs/__init__.py", line 139, in trampoline                                
     raise RuntimeError('do not call blocking functions from the mainloop')                                             
 RuntimeError: do not call blocking functions from the mainloop

this ligitemtirly does look like its a blockign read on the main loop but im not sure if its
an sqlalchemy bug, or something in howe we are using it.

what i dont really understand is how is that runnign on the mainlop rahther then a greentread where it will be monkey patched.

the orginal execption i was seeign are nologner present with the tip of master so i can confirm #1015 resolves this bug.

Can you open a new GitHub issue for the final issue that remains?

Navigation Menu

Repository navigation

time.sleep(0) or greenthread.sleep(0) is docuemnted as the canonical way of expressing a cooperative yield #1014

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	seconds may be specified as an integer, or a float if fractional seconds
	are desired. Calling :func:`~greenthread.sleep` with seconds of 0 is the
	canonical way of expressing a cooperative yield. For example, if one is
	looping over a large list performing an expensive calculation without
	calling any socket methods, it's a good idea to call ``sleep(0)``
	occasionally; otherwise nothing else will run.
	"""
	hub = hubs.get_hub()
	current = getcurrent()
	if hub.greenlet is current:
	raise RuntimeError('do not call blocking functions from the mainloop')

time.sleep(0) or greenthread.sleep(0) is docuemnted as the canonical way of expressing a cooperative yield #1014

Description

Activity

itamarst commented on Jan 15, 2025

SeanMooney commented on Jan 15, 2025

4383 commented on Jan 17, 2025

SeanMooney commented on Jan 17, 2025

itamarst commented on Jan 17, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions