Go Reflect: Fail to Set Argument Value During Conversion¶

This blog records the failure when trying to set the response value using reflect to provide an adaptor for wrapping the original PRC interceptor to the custom interceptor format defined by our tech products.

The difference between them is the custom interceptor moves the response from an argument to a response. This blog introduces the challenges encountered while designing an adaptor.

// package rpc
// original RPC interceptor signature
type Interceptor func(ctx context.Context, 
    request, response interface{}, processor Processor) uint32
type Processor func(ctx context.Context, request, response interface{}) uint32


// package custom
// interceptor in the tech team
type CustomInterceptor interface {
    Wrap(HandlerFn) HandlerFn
}
type HandlerFn func(ctx context.Context, req interface{}) (interface{}, error)

As the RPC framework requires to register services for commands, the original RPC interceptors receive interfaces with underlying concrete pointers to empty(default) structures, which is an inconsistent behavior from the origin and the custom signature.

The challenge of this adaptor is the underlying type and value of response interface:

• Pre-processor stage: the response type is lost as the HandlerFn truncates the response type.
• Post-processor stage: the response value should be set automatically by the adaptor.

However, both goals listed above failed because of the limitations of the current system design and reflect. This blog will record more details about the failures.

Analyzing Challenges¶

Given Up for Pre-Processor Stage¶

As the preface introduces the challenges, the custom framework signature cuts the response type/value off from the beginning. As a result, there is no way for the adaptor to handle such loss. Hence, we have no way but to give up and assume users will try to use the response type/value when they wrap their original interceptors by the adaptor.

Struggling in Post-Processor Stage¶

The challenge in the adaptor for the post-processor stage is that the response in the original interceptor should be set correctly by the adaptor called by itself. For a better understanding, what the adaptor should do is revealed by the pseudocode below.

func originalInterceptorToBeWrapped(ctx context.Context,
    request, response any, processor Processor) uint32 {
    // pre-processor stage:
    // if it's wrapped by the adaptor, the response is lost
    // because the custom framework truncates the value passed by RPC framework
    code := processor(ctx, request, response)

    // post-processor stage
    // the processor executes the real handler and get the response value
    // however, the processor MUST do some stuffs for the response
    // otherwise, it won't be changed ,
    // even the real response is got inside the processor
    return code
}

Implementation to Set Value¶

Following the idea mentioned above, we need to set the response value inside the adaptor. Hence, we need to construct a pointer to an interface so the adaptor could set the value inside it. The implementation looks like this:

type ServerAdapter rpc.Interceptor

func (sa ServerAdapter) Wrap(
    handlerFunc custom.CustomInterceptor) custom.CustomInterceptor {

    processor := func(ctx context.Context, request, response any) uint32 {
        realResp, err := handlerFunc(ctx, request)

        // response is an empty interface passed by the adapter
        // or an interface with an interface pointer
        // spiResp is the interface with concrete underlying type
        // we set response to pass the spiResp value and type into response
        original := reflect.ValueOf(response)
        original.Elem().Set(reflect.ValueOf(realResp))

        if err != nil {
            return convertErrToCode(err)
        }
        return 0
    }

    return func(ctx context.Context, req any) (any, error) {
        var resp any
        code := sa(ctx, req, &resp, processor)
        err := convertCodeToErr(code)
        return resp, err
    }
}

Error: Different Types Between Expected and Actual Types¶

As the implementation reveals, the response value could be set. However, it isn't set up correctly.

Hence, even though the final values(I mean, let's skip the pointers wrapped by interfaces) are the same, we will get a response with the wrong type compared to the expectation.

For example, illustrating with the response type EchoResponse:

type EchoResponse struct {
    Data             *string `protobuf:"bytes,2,opt,name=data" json:"data"`
}

-- // expected value:
-- interface { *EchoResponse }
++ // actual value:
++ interface { *interface { *EchoResponse } }

It's caused that once we pass the &resp into function sa, it will immediately be casted interface { *interface {nil} }.

        var resp interface{}
        code := sa(ctx, req, &resp, processor)

The casting fails the further setting up by reflect. Because Go passes all arguments by values, its type is interface { *interface { nil } } when we want to modify the response.

    processor := func(ctx context.Context, 
        request, response interface{}) uint32 {

        // response has a type of `interface { *interface { nil } }`
        original := reflect.ValueOf(response)
        original.Elem().Set(reflect.ValueOf(realResp))
        // ignore some lines
    }

Conclusion¶

Go interface will wrap a pointer to a new interface as its underlying pointer. Moreover, the interfaces are passed by values. Hence, the underlying pointers are non-settable by reflect as they are values.