Ошибка recv errno 11

I possess a couple of codes:
server.py
and
talk.py
.

The intended function of
server.py
involves receiving incoming connections and processing the information transmitted by
talk.py
.

The performance is flawless when running
server.py
and
talk.py
on separate terminals of the same machine, i.e., localhost.

In
talk.py
, a string is requested. This string is then consumed by
server.py
and printed in the terminal running the program as per
server.py
.

As previously stated, the scripts are functional on an Ubuntu machine that is local.

Upon deploying my
server.py
to a GCP VM instance, it appears to run smoothly. However, upon attempting to connect my
talk.py
code to the
server.py
socket through an external web IP address on my local machine, I encountered an error code from
talk.py
.

Traceback (most recent call last):
  File "talk.py", line 11, in 
    s.connect(('35.247.28.0', port))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 11] Resource temporarily unavailable

#server.py
# first of all import the socket library
import socket               
# next create a socket object
s = socket.socket()         
print "Socket successfully created"
# reserve a port on your computer in our
# case it is 12345 but it can be anything
port = 12345               
# Next bind to the port
# we have not typed any ip in the ip field
# instead we have inputted an empty string
# this makes the server listen to requests 
# coming from other computers on the network
s.bind(('', port))        
print "socket binded to %s" %(port)
# put the socket into listening mode
s.listen(5)     
print "socket is listening"           
# a forever loop until we interrupt it or 
# an error occurs
while True:
   # Establish connection with client.
   c, addr = s.accept()     
   print 'Got connection from', addr
   # Get data from client
   """
   try:
       data = c.recv(1024)
   except IOError as e:
       print e
   """
   print data
   if not data:
     break
   # Send back reversed data to client   
   c.sendall(data)
   # send a thank you message to the client. 
   #c.send('n Thank you for sending message!!!!')
   # Close the connection with the client
   c.close()

# talk.py
# first of all import the socket library
import socket               
# next create a socket object
s = socket.socket()         
print "Socket successfully created"
# reserve a port on your computer in our
# case it is 12345 but it can be anything
port = 12345               
# Next bind to the port
# we have not typed any ip in the ip field
# instead we have inputted an empty string
# this makes the server listen to requests 
# coming from other computers on the network
s.bind(('', port))        
print "socket binded to %s" %(port)
# put the socket into listening mode
s.listen(5)     
print "socket is listening"           
# a forever loop until we interrupt it or 
# an error occurs
while True:
   # Establish connection with client.
   c, addr = s.accept()     
   print 'Got connection from', addr
   print data
   if not data:
     break
   # Send back reversed data to client   
   c.sendall(data)
   # send a thank you message to the client. 
   #c.send('n Thank you for sending message!!!!')
   # Close the connection with the client
   c.close()

It appears that there may be an issue with routing or security, rather than a problem with your code. To address this, you could try port forwarding for the Google Compute Engine instance.

An additional source is available on the topic of opening a particular port, such as 9090, in Google Compute Engine.

To optimize the execution time of my script that comprises of 20,000 tasks involving subprocess and TCP calls, I’m utilizing the recently introduced
asyncio
feature in Python.

Although my script is running, the errors that Python outputs are causing me some worry.

Exception ignored when trying to write to the signal wakeup fd:
BlockingIOError: [Errno 11] Resource temporarily unavailable

The code will output multiple prints without throwing any errors. In the past, I faced issues such as
OSError
and lost connections to the servers. To resolve this, I implemented semaphores to restrict the number of connections to each server to 100 and a total of 700 connections.

As Python is not throwing any exceptions, the errors cannot be intercepted, although they do not appear to impact the script.

Should I worry about these errors? And if yes, how can I eliminate them from my program’s output? But, if not, what steps do I need to take to remove them?

Moreover, if the errors are critical, what is the reason behind Python ignore not raising an exception and just reporting them?

The bottleneck seems to be caused by running multiple brief
subprocess
sessions, as indicated by the python bug tracker data.

The error message «Exception ignored when trying to write to the signal wakeup fd» is generated by the signal handler located in Modules/signalmodule.c. This issue arises because the number of SIGCHLD signals received by Python is high, as the test scripts generate more than 300 processes each second on my computer. The producer, which writes the signal number into the «self» pipe, is faster than the consumer, which is the BaseSelectorEventLoop._read_from_self callback function.

After applying the patch, I receive messages showing 140 concurrent processes. This is a significant improvement. However, in my opinion, exceeding 100 concurrent processes is not advisable, especially for processes with brief lifetimes. The limitation arises from the number of SIGCHLD per second, which is determined by the number of processes that terminate at the same second.

To avoid the errors, I limited the number of
create_subprocess_exec
es that run concurrently by modifying the code. I observed that the errors vanished when the limit was set to less than 35, but I intend to lower it to 20 as an added precaution. However, the optimal number may differ, depending on the situation.

async def myTask(stuff, semaphore, loop):
    with semaphore:
        process = await asyncio.create_subprocess_exec('short_program', loop=loop)
def taskRunner(stuffs):
    loop = asyncio.get_event_loop()
    semaphore = asyncio.Semaphore(20)  # limit how many can run at a time
    tasks = [
        asyncio.ensure_future(myTask(semaphore, loop))
        for i in range(20000)
    ]
    loop.run_until_complete(asyncio.gather(*tasks))
    loop.close()

My web app is set up on Python 3.7 in a standard environment. During local testing, everything runs smoothly. But, when I attempt to deploy it, I encounter an error while the app is trying to save files to a specific location (
/tmp
) that I have assigned.

BlockingIOError: [Errno 11] Resource temporarily unavailable

I’m specifying both the location to save and the name of the file.

file_num = os.urandom(10).hex()
handle, path = tempfile.mkstemp()
ytdl_format_options = {'format': 'bestaudio/best','outtmpl':path + 'song'+ file_num +'.mp3', 'quiet': True}

Uncertainty arises as to the cause of the error and whether a configuration file setting on
app.yaml
or gunicorn was missed. Despite efforts to locate the gunicorn files on the glcoud shell, they were not found in the directory.

Upon deploying my app after making new changes to files, I have observed that it functions properly only for the first attempt. Subsequently, when I try running the function again, it fails and generates an error.

Traceback (most recent call last):
  File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 279, in handle
    keepalive = self.handle_request(req, conn)
  File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 348, in handle_request
    six.reraise(*sys.exc_info())
  File "/env/lib/python3.7/site-packages/gunicorn/six.py", line 625, in reraise
    raise value
  File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 331, in handle_request
    resp.write_file(respiter)
  File "/env/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 403, in write_file
    if not self.sendfile(respiter):
  File "/env/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 393, in sendfile
    sent += sendfile(sockno, fileno, offset + sent, count)
BlockingIOError: [Errno 11] Resource temporarily unavailable

From Filesystem:

The runtime has a complete filesystem that is mostly read-only, except for the virtual disk called
/tmp
. This virtual disk stores data in the RAM of your App Engine instance.

So:

• The utilization of any location besides
/tmp
is ineffective.
• Given the limited available space, it is crucial for your app to delete temporary files once they are no longer needed. Based on the behavior you have described, it seems that the available space may only be sufficient for storing one of your files.

When dealing with multiple parallel requests, it is important to take into account the possibility of the creation of multiple files. This can lead to errors if different requests attempt to write to these files simultaneously. To prevent this issue, it may be helpful to implement a scheme that limits the number of requests processed in parallel.

Though not completely confirmed, it seems likely that the limit could rely on the overall RAM of the instance. Therefore, the instance class set up for your app may also play a role. To verify this, experiment by temporarily setting up an instance class with more RAM to see if it resolves the issue. You could try this without clearing out the files, and it might work more than once before hitting the error.

If the aforementioned doubt is verified, opting for a suitable instance class may serve as a solution to tackle the issue.

Although all the mentioned solutions have been tried, encountering the issue is still a possibility. This may occur when the files being written are too large for the local filesystem to handle. In such a case, it is not suitable for your application. Instead, you can consider Google Cloud Storage (GCS) as an alternative for storing your files.

My approach for transferring data between two processes involves the use of a UNIX UDP socket that is nonblocking.

audio_s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
audio_s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 5500000)
audio_s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 5500000)
audio_s.setblocking(0)
try:
    os.remove(SOCKET_PATH)
except FileNotFoundError as e:
    pass
audio_s.bind(SOCKET_PATH)

Frequently, I transmit information to a process located remotely.

audio_s.sendto(data, REMOTE_SOCKET_PATH)

Occasionally, I come across
BlockingIOError: [Errno 11] Resource temporarily unavailable
during a call on
sendto
. It’s quite uncommon. I wonder what could be the reason for this. It seems like
Resource temporarily unavailable
is a very generic statement.

1. Can I obtain more specific details? For instance, can I transmit buffer overflowed to receive buffer overflow remotely? Are both options feasible or just one of them?
2. Is it possible for me to manage the number of datagrams in the send/receive buffer?
3. What is the proper approach to handle or prevent this scenario?

If I remove line with:

audio_s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 5500000)

it works without this exception

‘man sendto’ defines errno 11 as eagain.

The operation requested on the socket marked as non-blocking would result in blocking.

There are clear explanations available regarding the situation where a non-blocking send() only transfers some data. In such cases, is it safe to assume that EWOULDBLOCK error will be returned on the next call? Additionally, there is information about the EAGAIN error related to the usage of the Berkeley Socket API.