Cancel and Timeout Implementation in Http Standard Lib¶
This page is a series of Design Http Client And Comparing to Standard Lib, there it mainly discussed the design of http client by myself and field RoundTripper.
In this pessage, we will continue to look up the field timeout, also explore how the send/write could be controled by caller to avoid blocking in net/http.
Field: timeout¶
Timeout is requisite for making network request, or you might be pending there forever. The http client provides a timeout field. What’s more, we could put a deadline context inside request or setup a timeout context in the transport layer.
Here, I mainly talk about client layer timeout. The transport layer is too far from our users so except special requirement, I think we’d better not use it.
Client timeout and request with timeout context¶
This topic discusses the two ways to set up the timeout:
-
Cient timeout
-
Timeout context within a request
ctx, cncl := context.WithTimeout(context.Background(), time.Second*3)
defer cncl()
req, _ := http.NewRequestWithContext(ctx,
http.MethodGet, "https://google.com", nil)
resp, _ := http.DefaultClient.Do(req)
The reason to put them together without transport is both of them belong to the scope of http client. Instead, transport is a deeper layer which relates more about implementation.
The overall diragram should look like this:
Merge Context with Deadline¶
After checking function setRequestCancel, the timeout in client will be merged with request context if any like this:
if oldCtx := req.Context(); timeBeforeContextDeadline(deadline, oldCtx) {
req.ctx, cancelCtx = context.WithDeadline(oldCtx, deadline)
}
This means the shorter time will finally take effects on the request timeout. Here, only deadline and cancelCh occur here, the context is compared and override if the deadline is shorter.
Closure for future closing¶
Judging timeout and decide what to return is not the duty of setRequestCancel, instead, it starts a new go routine, which check three channels.
The stopTimerCh could be closed from outside, as it is returned in a closure.
func setRequestCancel(req *Request, rt RoundTripper, deadline time.Time) (stopTimer func(), didTimeout func() bool) {
if deadline.IsZero() {
return nop, alwaysFalse
}
knownTransport := knownRoundTripperImpl(rt, req)
oldCtx := req.Context()
if req.Cancel == nil && knownTransport {
// If they already had a Request.Context that's
// expiring sooner, do nothing:
if !timeBeforeContextDeadline(deadline, oldCtx) {
return nop, alwaysFalse
}
var cancelCtx func()
req.ctx, cancelCtx = context.WithDeadline(oldCtx, deadline)
return cancelCtx, func() bool { return time.Now().After(deadline) }
}
initialReqCancel := req.Cancel // the user's original Request.Cancel, if any
var cancelCtx func()
if oldCtx := req.Context(); timeBeforeContextDeadline(deadline, oldCtx) {
req.ctx, cancelCtx = context.WithDeadline(oldCtx, deadline)
}
cancel := make(chan struct{})
req.Cancel = cancel
doCancel := func() {
// The second way in the func comment above:
close(cancel)
// The first way, used only for RoundTripper
// implementations written before Go 1.5 or Go 1.6.
type canceler interface{ CancelRequest(*Request) }
if v, ok := rt.(canceler); ok {
v.CancelRequest(req)
}
}
stopTimerCh := make(chan struct{})
var once sync.Once
stopTimer = func() {
once.Do(func() {
close(stopTimerCh)
if cancelCtx != nil {
cancelCtx()
}
})
}
timer := time.NewTimer(time.Until(deadline))
var timedOut atomicBool
go func() {
select {
case <-initialReqCancel:
doCancel()
timer.Stop()
case <-timer.C:
timedOut.setTrue()
doCancel()
case <-stopTimerCh:
timer.Stop()
}
}()
return stopTimer, timedOut.isSet
}
func setRequestCancel(/*ignore arguments*/)
(stopTimer func(), didTimeout func() bool) {
// ignore lines
go func() {
select {
case <-initialReqCancel:
case <-timer.C:
case <-stopTimerCh:
}
}()
return stopTimer, timedOut.isSet
}
Why it returns a stopTimer function? From the implementation we could know it aims
if !deadline.IsZero() {
resp.Body = &cancelTimerBody{
stop: stopTimer,
rc: resp.Body,
reqDidTimeout: didTimeout,
}
}
When the timer.C happened and actually it call cancelRequest in a deep call stack, finally the connection will be closed by the request.
In the end, the cancelTimerBody will read a network error and report a timeout error.
Context check starts at RoundTripper¶
Previous, the context and deadline are compared and merged. In this topic, we enter the function RoundTrip:
When sending request, many places check the context. The http transport checks it first, and then enter the persist connection roundtripper, there checks context also.
Note that from here there is no deadline any more, it works as same as context if merged.
[http transport] check ctx before sending¶
Firstly, before calling persist connection roundtrip, http roundTrip checks the context in request:
func (t *Transport) roundTrip(req *Request) (*Response, error) {
ctx := req.Context()
for {
select {
case <-ctx.Done():
req.closeBody()
return nil, ctx.Err()
default:
}
// send data out by pConn roundtrip
}
[pConn] check and send in different go routine¶
The acticities in persist connection is shown in the diagram:
Here I only care about context, so it will cancel the request after passing req and response to the other go routines:
for {
case <-pcClosed:
pcClosed = nil
if canceled || pc.t.replaceReqCanceler(req.cancelKey, nil) {
if debugRoundTrip {
req.logf("closech recv: %T %#v", pc.closed, pc.closed)
}
return nil, pc.mapRoundTripError(req, startBytesWritten, pc.closed)
}
case <-ctxDoneChan:
canceled = pc.t.cancelRequest(req.cancelKey, req.Context().Err())
cancelChan = nil
ctxDoneChan = nil
}
}
It’s easy to debug for assure as the send is in another go routine. You could add a break point writeLoop to block the write, and then the timeout happenes inside pConn roundtrip.
- break point demo:
After testing, the timeout in client and the context carried in request both triggered the ctxDoneChan. The cancelch in request will trigger cancelChan. But they are essentially same, both of them cancel the request.
The cancelRequest is introduced below.
What does cancelling a request mean?¶
The content above cites if there is deadline or timeout, the request will be canceled. However, the request contains many parts, such as:
- a part of request is sent out(body is sent or not).
- whole request is sent but haven’t get a response.
- receive a part of response.
For those cases, are there a general behavior among them?
In the above diagram, we found that request and response channel are sent and will be handled by writeLoop and readLoop.
- A wired behavior During debug, there is a very interesting thing, first there are two imporant steps, write to buffer and flush in writeLoop:
err := wr.req.Request.write(pc.bw, pc.isProxy,
wr.req.extra, pc.waitForContinue(wr.continueCh))
if err == nil {
err = pc.bw.Flush()
}
Here, I added some break points and set timeout with 2 second, N/A means no break point. The sequence of a request：
| write | flush | result |
|---|---|---|
| stay 3s | N/A | write a closed connection |
| stay 1s | N/A | receive by the peer |
| N/A | stay 3s | receive by the peer |
| N/A | stay 1s | receive by the peer |
| N/A | 30s | receive by the peer |
However, the result is very wired as staying in write or flush should behavior same. However, the result refers a different case. Todo: track for unfinishing : explore more if having time, raised a discussion in golang-nut.
However, read the code, we could know the go routines of close the connection and write data have no order guarantee, the only shared resouce is the tcp connection.
func (w persistConnWriter) Write(p []byte) (n int, err error) {
n, err = w.pc.conn.Write(p)
w.pc.nwrite += int64(n)
return
}
func (pc *persistConn) closeLocked(err error) {
if err == nil {
panic("nil error")
}
pc.broken = true
if pc.closed == nil {
pc.closed = err
pc.t.decConnsPerHost(pc.cacheKey)
// Close HTTP/1 (pc.alt == nil) connection.
// HTTP/2 closes its connection itself.
if pc.alt == nil {
if err != errCallerOwnsConn {
pc.conn.Close()
}
close(pc.closech)
}
}
pc.mutateHeaderFunc = nil
}
In conclusion, the cancelling request means close the tcp connection, and clean something.
Different between client timeout and request timeout¶
Based on the anaysis above, we could know the client timeout take a further step than request timeout. The request timeout ends when the response is done to construct, however, the client timeout could still take effects when the reponse returned as it carries a manuplated body. The picture below shows well, refer to the article The complete guide to Go net/http timeouts.
We could test it easily.
- Server side, sleep 1s and write data twice.
func (s *server) ServeHTTP(w http.ResponseWriter, req *http.Request) {
time.Sleep(1 * time.Second)
_, err := w.Write([]byte("hello"))
fmt.Printf("receive request: %s, write err: %s\n", time.Now(), err)
time.Sleep(1 * time.Second)
_, err = w.Write([]byte("world"))
fmt.Printf("receive request: %s, write err: %s\n", time.Now(), err)
}
-
[Client]: get EOF when using request context timeout
func main() { cli := http.Client{} req, _ := http.NewRequest(http.MethodPost, "http://127.0.0.1:8080", nil) ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) defer cancel() req.WithContext(ctx) res, err := cli.Do(req) fmt.Printf("send request: %s\n", err) vs := make([]byte, 10) _, err = res.Body.Read(vs) fmt.Printf("%s, err: %s", vs, err) } -
[Client]: context deadline exceeded (Client.Timeout exceeded while awaiting headers) error and panic when reading body
The reason why is that once the client timeout happenes, the timer.C in another go routine will close the connection, as a result, the response is invalid right now.
Body in request and response¶
Request body¶
Request body will be closed if it really has a Close method based its construction, refer to the NewRequestWithContext:
// body typed io.Reader
rc, ok := body.(io.ReadCloser)
if !ok && body != nil {
rc = io.NopCloser(body)
}
If the io.Reader doesn’t implement Closer interface, it is wrapped by io.NopCloser.
When we write the http request body to the writter, it will check and close the request. By the way, source code also whether it’s a file for optimization, refer the comment here.
Response body¶
Let’s see how the response is constructed first. It’s created by readTran, which wraps the tcp connection.
It wraps the pc.br, which is a bufio.Reader from connection, it’s assigned when dail:
This means read body actually read from the tcp connection.. If we not close the body when we read from it, the connection will pending there and won’t be cleared.
- Construct the body:
switch { case t.Chunked: if noResponseBodyExpected(t.RequestMethod) || !bodyAllowedForStatus(t.StatusCode) { t.Body = NoBody } else { t.Body = &body{src: internal.NewChunkedReader(r), hdr: msg, r: r, closing: t.Close} } case realLength == 0: t.Body = NoBody case realLength > 0: t.Body = &body{src: io.LimitReader(r, realLength), closing: t.Close} default: // realLength < 0, i.e. "Content-Length" not mentioned in header if t.Close { // Close semantics (i.e. HTTP/1.0) t.Body = &body{src: r, closing: t.Close} } else { // Persistent connection (i.e. HTTP/1.1) t.Body = NoBody } }
Conclusion¶
For request, we needn’t to care about closing it any more. But for server, we need to close it or it causes a resource leak. The connection won't be re-used, and can remain open in which case the file descriptor won't be freed.
What should client do if timeout?¶
From the content above, we know client return when timeout no matter the request are sent, response received or not.
It’s not the duty of client to make sure what happened when timeout, instead, it’s the duty of server. Server needs to guarantee the mechanism, eg only proceed the data once, to avoid client timeout and retry makes side effects.
For example, if you modify the mysql, a database transaction might help you to avoid such issue. Mutex in applcation layer if there is only one instance. Redis might also help the server.