[e2e] end2end-interest Digest, Vol 83, Issue 4

[Zama Ques]

Hi Yan,

Thanks for your suggestion .  I am familiar with iperf but the issue with us that it is a prod network and it is advisable for me not to pump data on the network . Will try to the experiment between two desktops connected by a cross over cable. 

What I was trying earlier was that I started FTP server on one end  and connected to the server from the client side. 

 $ ftp 10.66.X.X
Connected to 10.66.X.X
220 (vsFTPd 2.2.2)
Name (10.66.74.141:zama): anonymous
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.


After that I disconnected the network cable from the server and was monitoring the status of the connection on the client side .
The status of the connection was like this before and after disconnecting the network cable. 

---
$  for i in {1..1000} ; do netstat -at | egrep "ftp" ; date  ; sleep 60  ; done
tcp        0      0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED 
Wed Feb 23 11:47:53 IST 2011

tcp        0      0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED 
Wed Feb 23 11:48:53 IST 2011
tcp        0      0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED 
Wed Feb 23 11:49:53 IST 2011
...
...
Wed Feb 23 12:14:03 IST 2011
tcp        0      0 edgebeauty.c:50179 shopfrobsoon.c:ftp ESTABLISHED 
Wed Feb 23 12:15:03 IST 2011
===

If we see that the time is more than 25 minutes when the server went down and the client has still maintained the connection in established state. 

My understanding is that the client should close the connection after TCP restarsmit timeout happens or my understanding is wrong. 

Please clarify . 

--Zaman


Message: 2
Date: Tue, 22 Feb 2011 09:55:13 -0500
From: Yan Cai <ycai at ecs.umass.edu>
Subject: Re: [e2e] query on behaviour of tcp_keepalive and tcp
    retransmit on    Linux based systems
To: end2end-interest at postel.org
Message-ID: <4D63CE50.8050606 at ecs.umass.edu>
Content-Type: text/plain; charset="iso-8859-1"

Hi

According to your description, the expected behavior should be as follows.
At the beginning senders at one side can send data to the receivers at 
the other side, and the receivers can receive data without any problem. 
When some of the receivers become off-line, the affected senders should 
no long receive positive acknowledgments, therefore, lowering their 
congestion windows (i.e., sending rate). Since in your case the receiver 
is off forever, some senders should further experience timeout events. 
After a few timeouts, the sender should CLOSE this connection itself.

As far as I know, the whole procedure above should be automatically 
invoked in the sender side. This is how TCP (sender) handles exceptions.

My suggestion is that you run a simple experiment on your side to see if 
TCP in your machine can work that way. The test can be done using i-perf 
to send a long long live TCP flow, and then take off the receiver in the 
middle of the transmission. The connection is expected to be closed very 
soon after the receiver is off.

Hope it helpful.
Yan
On 2/22/2011 4:24 AM, Zama Ques wrote:
> We need some clarifications on TCP_keepalive .  We are facing some 
> issues on our Prod servers related to TCP functionality .
>
> The issue is like this.
>
> We have some machines at one end sending data in real time to another 
> group of machines on the other hand .  Now due to some hardware issues 
> on the other hand , some of the machines becomes unresponsive/crashes. 
> The client system which pumps data never came to know that the server 
> went unresponsive . The connection remains in
> ESTABLISHED state and the client always tries to send data thinking 
> that the connection is alive because of which we are seeing backlog on 
> client sides.
>
> Our understanding is like this on how TCP will handle the connection.
>
>
> Q 1) Since  the server went down , the client will try to the 
> retransmit the data until it times out. What is the behavior of TCP 
> after the timeout? Need clarification on
> the following things.
> a) Will the kernel will close the established connection after the 
> timeout . Looks like no in our case as we still see the connection 
> still in ESTABLISHED state after around more
> than 2 hours.
> b) Are there any kernel parameters which decides the when the client 
> is timeout after retransmission fails. What is the behavior of TCP 
> after the client retransmission timeouts.
>
>
> Q 2 ) There is something called tcp_keepalive which if implemented in 
> the kernel , by default it's there and comes to be around 2 hrs 2 
> minsutes , i think  ,  the client will send some TCP probes after the 
> keepalive time ineterval and if it cannot reach the server , then the 
> established connection in the client side will be closed by the kernel 
> . This is my understanding. But I can see that the connection still 
> remains in established after the tcp_keepalive time . We waited for 
> around 2 hrs 30 minutes but the connection remains in established 
> state only. Tried reducing the keepalive time to be around 10 minutes 
> , but the connection remains in ESTABLISHED state in client side .
>
>
> Where I went wrong .Please clarify my doubts raised above . What 
> should we do to resolve the problem we are seeing above . Any help 
> will be highly appreciated as we are going through a hard time to 
> resolve the issue .
>
> Thanks in Advance
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110222/50be8540/attachment-0001.html

------------------------------

_______________________________________________
end2end-interest mailing list
end2end-interest at postel.org
http://mailman.postel.org/mailman/listinfo/end2end-interest


End of end2end-interest Digest, Vol 83, Issue 4
***********************************************


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20110223/e1f94dc0/attachment.html