What can I do to configure SSH on both client and servers to prevent Write Failed: broken pipe errors? It often occurs if you sleep your client computer and resume later.
LiveWireBT
28.4k26 gold badges107 silver badges219 bronze badges
asked Apr 28, 2012 at 23:36
11
I have tried this in /etc/ssh/ssh_config for Linux and Mac:
Host *
ServerAliveInterval 120
This is how often, in seconds, it should send a keepalive message to the server. If that doesn’t work then train a monkey to press enter every two minutes while you work.
You could set either ServerAliveInterval in /etc/ssh/ssh_config of the client machine or ClientAliveInterval in /etc/ssh/sshd_config of the server machine. Try reducing the interval if you are still getting the error.
Configuration for a single user can be set in file ~/.ssh/config both on the server and client side. Make sure the file has correct permissions chmod 644 ~/.ssh/config.
answered May 26, 2012 at 11:49
Aram KocharyanAram Kocharyan
3,6802 gold badges15 silver badges8 bronze badges
19
SSH sessions may break due to numerous and possibly unavoidable reasons.
A useful utility which can be used to mitigate problems caused by this is called screen. Screen is a powerful utility that allows you to control multiple terminals which will stay alive independently of the ssh session. For example, if you run screen in an ssh session you will see a new terminal open and you can use that to run jobs. Lets say your ssh session dies in the process. Running screen -d then screen -r will reopen the last session and you will be able to continue from there. Make sure you read some of the documentation before using it.
answered Oct 4, 2012 at 16:28
eltommoeltommo
1,9041 gold badge12 silver badges11 bronze badges
9
Client configuration
Try creating the file:
~/.ssh/config
Add the contents:
Host *
ServerAliveInterval 30
ServerAliveCountMax 5
Now ssh to your server and see if your problem is fixed. ClientAliveInterval option is only useful when configuring the ssh server (aka sshd), it does not change a thing on the ssh client side, so don’t use it in the above configuration file.
This will send a hello-are-you-there signal to the server if no packets have been received in the preceding 30 seconds (as specified above). However, if the number of consecutive hello-are-you-there signals reach ServerAliveCountMax then ssh will disconnect from the server. This value is defaulting to 3 (so 3*30 = 90 seconds without server activity), increase it if it suits your needs. There are alot more config options to the .ssh/config file and you could read:
Using an SSH Config File
For more information on other options. You may not want to apply this to every server you connect to which this example will. Or restrain it to only a particular server by replacing the line Host * with Host <IP> (replace by an IP address, see ssh_config man page).
Server configuration
Similarly you can tell the server to be gentle with your clients. The configuration file is /etc/ssh/sshd_config.
ClientAliveInterval 20
ClientAliveCountMax 5
You can either deactivate it by setting ClientAliveInterval to 0 or tweak ClientAliveInterval and ClientAliveCountMax to set a maximum ssh client inactivity without responding to the probes. One advantage of this settings over TCPKeepAlive is that the signals are sent through the encrypted channels, so it is less likely to be spoofable.
answered Oct 6, 2013 at 2:54
MattMatt
6925 silver badges4 bronze badges
9
I’m remotely upgrading an Ubuntu server from lucid to precise and lost the ssh connection in the middle of the upgrade with the message «Write failed. Broken pipe». ClientAliveInterval and ServerAliveInterval did nothing. The solution is to turn on TCPKeepAlive options in client ssh:
TCPKeepAlive yes
in
/etc/ssh/ssh_config
Hope this will help
answered Oct 7, 2012 at 18:40
For the client, edit your ~/.ssh/config (or /etc/ssh/ssh_config) file as follow:
Host *
TCPKeepAlive yes
ServerAliveInterval 120
TCPKeepAlive — Specifies whether the system should send TCP keepalive
messages to the other side. If they are sent, death of the connection
or crash of one of the machines will be properly noticed. However,
this means that connections will die if the route is down temporarily,
and some people find it annoying (The default is ‘yes’).ServerAliveInterval — Sets a timeout interval in seconds after which
if no data has been received from the server, ssh(1) will send a
message through the encrypted channel to request a response from the
server. The default is 0, indicating that these messages will not be
sent to the server.
For the server, edit your /etc/ssh/sshd_config as:
ClientAliveInterval 600
ClientAliveCountMax 0
If you want ssh client to exit (timeout) automatically after 10 minutes (600 seconds).
ClientAliveCountMax – This indicates the total number of checkalive
message sent by the ssh server without getting any response from the
ssh client. Default is 3.ClientAliveInterval – This indicates the timeout in seconds. After x
number of seconds, ssh server will send a message to the client asking
for response. Deafult is 0 (server will not send message to client to
check.).
See also: What do the options ServerAliveInterval and ClientAliveInterval in sshd_config do, precisely?
answered Oct 2, 2014 at 13:52
kenorbkenorb
9,8862 gold badges76 silver badges90 bronze badges
1
I absolutely love Mosh. I frequently ssh into a server, close my laptop and go to a cafe, open it up and carry on as if nothing changed.
Mosh (mobile shell)
Remote terminal application that allows roaming, supports intermittent
connectivity, and provides intelligent local
echo and line editing of user keystrokes.Mosh is a replacement for SSH. It’s more robust and responsive,
especially over Wi-Fi, cellular, and long-distance links.Mosh is free software, available for GNU/Linux, FreeBSD, Solaris, Mac OS X, and Android.
answered Sep 30, 2014 at 18:48
JakeJake
3012 silver badges3 bronze badges
For me, I was getting Write failed: Broken pipe even when I was actively typing in vim or at the shell prompt. I couldn’t browse the internet locally either for awhile either. (I was connecting remotely to Ubuntu using Terminal.)
Others in my network stream a lot of video from Netflix and other places. I can’t prove it, but I suspect its an ISP or router issue. For example, Verizon and Netflix are pointing fingers at each other for their customer’s network problems.
If you’ve got a dial-up connection and are streaming video or music with a simultaneous SSH or telnet connection, it’s inevitable at some point you’ll get a broken pipe message. Upgrading my ISPs broadband package seemed to make my broken connection less frequent.
answered Jul 30, 2014 at 13:33
pmaguniapmagunia
1,3721 gold badge10 silver badges14 bronze badges
answered Jan 25, 2019 at 3:13
rupert160rupert160
1711 silver badge2 bronze badges
I have a script on the remote server that never seems to fails, regardless of the SSH configuration client or server.
#!/bin/bash
while true; do date; sleep 10; done;
Save it to some dummy.sh file and quickly run it before you minimize the window or move away from it. It will keep printing the current time stamp on the server and keeps your connection alive as long as the connection is not dropped by any other reason. When you get back to that terminal, just hit CTRL+C and keep working.
answered Sep 9, 2013 at 16:50
JulioHMJulioHM
6341 gold badge5 silver badges11 bronze badges
2
You can add these args every time you invoke ssh: -o ServerAliveInterval=15 -o ServerAliveCountMax=3
You don’t have to edit /etc/ssh/*config files if you do this.
You can create an bash alias or function or script to make this easy.
E.g. these bash functions, you can add into your .bashrc, do_ssh is used manually to turn on keepalives. do_ssh_pty is used within scripts to set pty and avoid prompts.
do_ssh() {
ssh -o ServerAliveInterval=15 -o ServerAliveCountMax=3 $*
}
do_ssh_pty() {
ssh -tt -o "BatchMode=yes" -o "StrictHostKeyChecking=no" -o ServerAliveInterval=15 -o ServerAliveCountMax=3 $*
}
Now do_ssh user@host can be used or do_ssh user@host <args> <command> and keepalives will be active.
answered Feb 12, 2018 at 13:05
gaoithegaoithe
5114 silver badges10 bronze badges
1
Updated on 5/5/2022 – Have you ever encountered a situation in which you were unable to download any package on your Linux machine ? Or you might have probably seen an error like package not installed? This kind of error can easily be fixed with a command like “sudo apt install –f”. On rare occasions, you may have experienced a broken pipe error.
A pipe in Linux / Unix connects two processes, one of them has read-end of the file and the other one has the write-end of the file. When a process writes to a Pipe, it gets stored in a buffer and gets retrieved by the other process. Broken pipe occurs when a process prematurely exits from either end and the other process has not yet closed the pipe.
Example use case:
A user has just recently reinstalled RVM (Ruby Version Manager) after he performed a fresh install of Ubuntu.
He then opened up the terminal and issued the command:
type rvm | head -1
This issued the following error:
rvm is a function -bash: type: write error: Broken pipe
What happened here is that when the user runs the command type rvm | head -1, bash has actually executed type rvm in one process and head -1 in another process. The stdout of the type part is connected to the “write” end of a pipe whereas the stdin of the head part is hooked up to the “read” end. Note that the two processes have run concurrently ( at the same time ).
The head -1 process has carried out a read operation of data from stdin , then prints out a single line (as dictated by the -1 option) before exiting, causing therefore the “read” end of the pipe to be closed. Since the rvm function has quite a long data stream (about 11 kB after having been bash parsed and reconstructed), which means that head exits yet at the same time type still has some data to write out (few KB).
Since type is trying to carry out a write operation to a pipe whose other end has therefore been closed – a brokenpipe routine or the write() function that it invoked, will return an EPIPE error which is known as “Broken pipe”.
Inspecting the Command
In most cases, this might not be the case but the first step you should check is whether the command issued was right or not. You should reissue the command and check whether it gets executed or not. You can also try issuing commands like “sudo apt update” and “sudo apt install –f” as these commands are not destructive in nature. If your problem still persists, try rebooting the machine and see whether the problem was resolved or not.
Fixing a Problem with File System
When you have issued the commands mentioned earlier multiple times and you still get the error, check whether the error reads something like “read-only file system” in the terminal output. This may be caused when your boot partition gets mounted as read-only for some reason. The problem could be caused by some faulty software installation when the system decides that it is not safe to write to the drive.
Read: How to create a Sudo user on Ubuntu
The other cause might be when you try to install something from apt and the installer needs to access some resource in read mode, but cannot perform the read operation properly. It may throw an error like “sudo: cannot mount”. This error occurs because most of the ‘entities’ in Linux are files and in order to read a resource, Linux would need to open that file and read it. If however another process is currently using that resource, then it may not be possible to read the file. Also, when the reading process exits abruptly and does not close the file, it may corrupt the file until the next boot.
If you still cannot access the files even after rebooting, then the problem could be bigger than anticipated. You may have a broken file system. To resolve this issue, you may need a stable Linux environment in order to work on the broken system. The best way to do this is to boot from a Live Linux USB drive and work from it.
This is the right moment to backup all your data. Although the following steps are safe, you should make sure to store your data on a secure device.
Once you boot into a Live USB drive you should start to check for the partitions with a corrupt file system. To do so, issue the following command:
sudo fsck.ext4 -fv /dev/sdaX”
Note that here X stands for the partition that you are trying to scan. Note that this command is for partitions of type ext4. If you have a partition of type ext3 or ext2 you will need to replace the command with “fsck.ext3” and “fsck.ext2” respectively. This will scan your drive and print the output on the terminal (note the -v flag). Alternatively, you can specify a -c flag to surface-scan the drive; it will look for bad sectors on the drive.
Once you have done this, your partition should hopefully been fixed. Now Boot into your machine and issue the command:
sudo mount -o rw,remount /
This will restore the read/write permission on your drive and will therefore solve the broken pipe issue.
Read: How to manage permissions in Linux : guide for beginners
Afterwords
You have just seen one solution to resolve the broken pipe issue, but a broken pipe is not a problem, it could be a symptom of a larger problem more often than not. You can have a broken pipe whenever you are trying to read a stream like resource that gets closed prematurely..
If you like the content, we would appreciate your support by buying us a coffee. Thank you so much for your visit and support.
If you use SSH to connect to remote Linux servers, you’ll notice that if you keep your SSH session inactive for some time and then try to use it again, the SSH session disconnects with an error message like this:
:client_loop: send disconnect: Broken pipeOn some systems, it will display ‘Write failed: Broken pipe’ or ‘Connection closed by remote host’.
Let’s see what causes this error and how to go about keeping your SSH connection alive.
Fixing broken pipe error with SSH
As you may have guessed, the SSH connection is closed because of inactivity. There is no set value but it usually around 5 minutes or so.
What you can do to avoid the SSH session disconnection is to send an ‘alive message’ either from the server to client (ClientAliveInterval) or from client to server (ServerAliveInterval) at certain time interval.
This way, you keep the SSH session alive because there is a communication between the client and server and the server understands that client is still there.
Now, there are two ways to do that. Either you send the alive message from the client to the server or from the server to the client.
- If you connect to multiple servers via SSH, set it on your machine.
- If you are a sysadmin and several of users complain about frequent SSH connection disconnect, you may set it on the server.
Method 1: Client side SSH configuration change
Let’s say you want to keep your SSH connection alive with up to 10 minutes (600 seconds) of idle time.
While connecting to the remote Linux system through SSH, you can mention the ServerAliveInterval value like this:
ssh -o ServerAliveInterval=600 [email protected]_ip_addressNow, this thing work but manually entering this option each time you connect to the server is tiresome. Why not make it permanent?
I hope you are aware of the SSH config files. On the client side, you can take advantage of it to set certain SSH parameters for specific connections or all of them. I have explained SSH config file in detail here.
First, make sure that you have the ssh config file. If not create it:
touch ~/.ssh/configIt is important to give it the correct file permissions otherwise you’ll have permission denied error while connecting via SSH.
Use the chmod command and add the following file permission to it:
chmod 600 ~/.ssh/configIf you’re feeling lazy or don’t want to go in detail, use this command to set the alive interval to 600 seconds (10 minutes):
echo "ServerAliveInterval 600" >> ~/.ssh/config This will set the ServerAliveInterval value to 10 minutes for all SSH connection you’ll use. Give it a try if you want to.
If you want to make it more proper, you should add it like this:
Host *
ServerAliveInterval 600Method 2: Server side SSH config change
The SSH config file for the server is usually located at /etc/ssh/sshd_config.
If you open this file, you’ll find two parameters of interest here:
ClientAliveInterval: This is the inactivity time period after which the server will send an alive message to the ssh connected client.ClientAliveCountMax: This is the number of attempts the server will make to send the alive message.
Say, you set ClientAliveInterval to 200 seconds and ClientAliveCountMax to 3. This means the server will send alive message after 200 seconds. If there is no activity from the client, it will again send an alive message at 400 seconds. No response/activity from the client and another alive message is sent at 600 seconds. After this (600 seconds) the SSH connection is disconnected.
You can edit the /etc/ssh/sshd_config file in your favorite terminal based text editor like Vim. Look for ClientAliveInterval and ClientAliveCountMax entries. Remove the # key at the beginning of the lines and give them the appropriate value.
Save and exit the file.
Please do not set the SSH connection timeout to several hours. That would be a waste of resources.
I hope this tutorial helped you to fix the broken pipe error issue with SSH connection. Your feedback is welcome.
Sometimes my SSH session disconnects with a Write failed: Broken pipe message. What does it mean? And how can I keep my session open?
I know about screen, but that’s not the answer I’m looking for. I think this is a sshd config option.
asked Sep 14, 2010 at 9:04
Peter StuifzandPeter Stuifzand
1,9712 gold badges12 silver badges7 bronze badges
It’s possible that your server closes connections that are idle for too long.
You can update either your client (ServerAliveInterval) or your server (ClientAliveInterval)
ServerAliveInterval
Sets a timeout interval in seconds after which if no data has
been received from the server, ssh(1) will send a message through
the encrypted channel to request a response from the server. The
default is 0, indicating that these messages will not be sent to
the server. This option applies to protocol version 2 only.
ClientAliveInterval
Sets a timeout interval in seconds after which if no data has
been received from the client, sshd(8) will send a message
through the encrypted channel to request a response from the
client. The default is 0, indicating that these messages will
not be sent to the client. This option applies to protocol
version 2 only.
To update your server (and restart your sshd)
echo "ClientAliveInterval 60" | sudo tee -a /etc/ssh/sshd_config
Or client-side:
echo "ServerAliveInterval 60" >> ~/.ssh/config
answered Sep 14, 2010 at 9:56
5
If you want to have a longer connection period, in the client add:
echo 'ServerAliveInterval 30' | tee -a ~/.ssh/config
echo 'ServerAliveCountMax 1200' | tee -a ~/.ssh/config
ServerAliveCountMax by default this is set to 3. Therefore once the ServerAliveInterval has sent 3 small packs of info to your server it will then automatically log out. Setting it to 1200 means this process will have to occur at least 1200 times. In short you should be connected at least 30*1200 seconds (10 hours).
answered Jul 4, 2015 at 22:40
5
An alternative solution would be to use mosh — the mobile shell. In contrast to ssh it connects via UDP and supports roaming. You can start your session at home, suspend your laptop, take it to work / friends / wherever else you have internet, unsuspend your laptop and continue to work as if nothing has happened. It is especially useful if you are on a lousy internet connection: It shows instant feedback if your keystrokes don’t reach the server and continuously tries to reestablish the connection.
Installation and setup are simple: It is now included in all current Linux (plus a few non-Linux) distributions and it coordinates the session initialization and authentication via a prior ssh connection. So if you are able to connect via ssh user@server you are very likely to be able to connect with mosh just by calling mosh user@server, if the mosh packages are installed on both ends.
The main reason for connection failures is that you have to reach the server on a UDP port (default range: 60000-61000) for mosh to work. So if the server is behind a firewall you are mostly out of luck if can’t punch holes in it yourself (security implications).
answered Aug 11, 2014 at 8:26
PerseidsPerseids
4635 silver badges11 bronze badges
I spent lots of time messing with ServerAliveInterval and ClientAliveInterval variables with no joy.
Finally, after comparing with a machine that did not have the broken pipe problem, I found the directive at the bottom of /etc/ssh/sshd_config:
KeepAlive yes
Putting this at the bottom of my problem server’s /etc/ssh/sshd_config did the trick.
Verified for Ubuntu 20.04.1 LTS, Ubuntu 16.04.7 LTS.
answered Jan 6, 2021 at 23:38
NormanNorman
611 silver badge1 bronze badge
3
It usually means that your network (TCP) connection was reset. E.g. your internet provider reconnected you or something like this.
answered Sep 14, 2010 at 9:44
maxschlepzigmaxschlepzig
55.8k48 gold badges202 silver badges276 bronze badges
0
I had the same problem but it is not as expected. If you find that on the same network another server is trying for the same IP address, you will face the same problem.
To solve this you need to check if there are other servers which use your same IP address. This can be done using the arp command.
I am using Debian so here is a sample of the commands that I use to determine if another server was indeed using the same IP address
apt-get install arp-scan
arp-scan -I eth0 -l | grep 192.168.1.42
192.168.1.42 d4:eb:9a:f2:11:a1 (Unknown)
192.168.1.42 f4:23:a4:38:b5:76 (Unknown) (DUP: 2)
You will notice two sets of mac address using the same IP address. Avoid conflict by setting one to another IP address.
steve
21.5k5 gold badges48 silver badges75 bronze badges
answered Dec 29, 2015 at 8:56
Another cause for the «Broken Pipe message» is that another machine is attempting use the same IP as your host.
A simple way to test if someone else is using that IP:
- Turn off your host
- ping the same IP to see if another machine is using that IP
To find out which machines are on your network, you can use this Unix & Linux question title: How to find what other machines are connected to the local network.
answered Apr 26, 2013 at 21:06
JosirJosir
1303 bronze badges
1
You can use SSH/SCP argument ‘-o‘ to accomplish that, no need to set your (if you’re admin) server’s ssh-config.
SSH options related to keeping connection alive are
TCPKeepAlive,ServerAliveCountMax,ServerAliveInterval.
Basically, it goes like this:
% ssh -o TCPKeepAlive=yes
-o ServerAliveCountMax=20
-o ServerAliveInterval=15
user@server.example
, to have the client sending «keep-alive» messages every 15 seconds until the limit of 20 messages — which amounts to 20 x 15 seconds = 5 minutes —, at which point the connection is considered broken/dead. The client process ends.
The man pages will give you more, detailed information:
- https://linux.die.net/man/5/ssh_config
answered Jul 23, 2020 at 8:56
BrandtBrandt
2821 gold badge2 silver badges10 bronze badges
For me this was because I had used ChrootDirectory which requires that the entire path given be owned by root (which it was not for me). Changing the permissions for the path and restarting sshd fixed the problem.
answered Feb 2, 2021 at 11:58
QwertieQwertie
1,1943 gold badges13 silver badges28 bronze badges
2
You must log in to answer this question.
Not the answer you’re looking for? Browse other questions tagged
.
Not the answer you’re looking for? Browse other questions tagged
.
SSH, an acronym for Secure Shell, is a remote network protocol that is used to securely connect to remote devices such as servers and network devices over a TCP/IP network.
It is a cryptographic network protocol that provides strong encryption technologies and hashing to secure communication between two devices on a network.
SSH uses two main authentication methods: password authentication and public key authentication. With password authentication, a user provides the remote host’s IP address or FQDN (Fully Qualified Domain Name) and password to authenticate.
Public key authentication uses an SSH key pair for authentication, which comprises two SSH keys: private and public keys.
The private key resides on the user’s machine and should always be kept confidential and secure. The public key is saved on the remote host that a user connects to. During authentication, the identity of the two keys is compared and access is granted.
When connecting to a remote system via SSH, you might encounter the error Client_loop: send disconnect: Broken pipe.
In this tutorial, we will see why this happens and address the error.
The error is simply a disconnection message that notifies you that your SSH connection timeout has been exceeded.
This is a period of inactivity during which no Linux command is executed or issued from the client side. When this happens, the SSH session is terminated, effectively disconnecting you from the remote server.
Most users will usually press ‘ENTER’ or a key on the keyboard to avoid having an idle SSH session which will cause the disconnection to the host. However, this can tedious and time-wasting.
Thankfully, SSH default configuration settings provide a few parameters that you can configure to keep your SSH connections active for longer periods of time.
Fix Client_loop: send disconnect: Broken pipe Error
To resolve this issue, you need to increase the SSH connection timeout on the client. To do so, modify the default SSH configuration file which is usually at /etc/ssh/sshd_config.
$ sudo vi /etc/ssh/sshd_config
Be sure to locate these two parameters: ClientAliveInterval and ClientAliveCountMax. Let’s check out what they do.
- ClientAliveInterval – This is the period of inactivity after which the SSH server sends an alive message to the remote client that is connected to it.
- ClientAliveCountMax – This is the number of attempts that the server will make to send the alive message from the server to the client.
We will set the two values as follows:
ClientAliveInterval 300 ClientAliveCountMax 3
This means that after the first 300 seconds (5 minutes) of inactivity from the client, the server will send an alive message to the client to keep the SSH session active.
If no data or response is received from the client for the next 300 seconds (at the 600-second mark), the server will again send another alive message. Finally, after 900 seconds of inactivity from the client, the SSH connection will be terminated or dropped.
Be sure to save the changes made to the file and then exit. Then restart the SSH daemon.
$ sudo systemctl restart sshd
Alternatively, you can connect to your remote client Linux system by specifying the ServerAliveInterval parameter in seconds (300 seconds), which means your SSH session is active for up to 5 minutes.
$ ssh -o ServerAliveInterval=300 [email protected]_ip_address
In this tutorial, we demonstrated how to resolve the Client_loop: send disconnect: Broken pipe error. As you have seen all you need is to perform a few tweaks in the SSH configuration file.