Module 5a - Apache Thrift

Question 1

What are the 4 internal software layers involved in a thrift service? What are they for?

Answer 1

1. Server - receives incoming connections
2. Processor - reads/writes I/O streams
3. Protocol - encodes/decodes data (can be into JSON)
4. Transport - reads/writes network (can use TCP, or HTTP)

Question 2

Give an example of a thrift IDL (.thrift file) for an interface that calculates the square root of a number, and throws an exception

Answer 2

service MathService {
   double sqrt(1: double num) throws (1: IllegalArgument ia)
}

exception IllegalArgument {
1: string message;
}

Question 3

What are the 9 base types for the thrift IDL?

Answer 3

bool
byte
i16
i32
i64
double
binary
string
void

Question 4

What are the 3 types of datatype containers in the thrift IDL?

Answer 4

list
set
map

Question 5

What are the other language constructs that thrift provides in its IDL?

Answer 5

const
typedef
enum
struct
exception

Question 6

What are the field modifiers for function signatures that thrift provides in its IDL?

Answer 6

required
optional
default values

Question 7

What is the “oneway” keyword used for in the thrift IDL?

Answer 7

building oneway thrift RPCs

Question 8

What does the service keyword define in the thrift IDL?

Answer 8

A service consists of a set of named functions, each with a list of parameters and a return type

Question 9

What does the “namespace” keyword determine in the thrift IDL?

Answer 9

Determines the java package

Question 10

After the IDL has been defined in a thrift file (lets call it file.thrift), what command is used to generate the source code files for the services?

Answer 10

thrift –gen file.thrift

Question 11

What steps must a synchronous thrift RPC client program execute when calling a service handler and invoking an RPC from the server?

Answer 11

Client must:
1. create a TSocket object using the network address of the server

2. Create a TTransport object using the TSocket object from step 1
3. Create a TProtocol object using the TTransport object from step 2
4. Create the service handler object from the TProtocol object in step 3. The client can now call the server’s service handlers and invoke the RPC

Question 12

What steps must a single-threaded thrift RPC server program execute when setting up a server which allows clients to invoke RPC calls to it?

Answer 12

1. Create a Processor object from the generated service handler files
2. Create a TServerSocket object using a port number to bind to
3. Create objects for protocolFactory, transportFactory, and processorFactory. The processorFactory will be the processor object from step 1
4. Create the TServer object from the arguments in step 3.
5. run the blocking “serve()” call

Question 13

What is the purpose of having the same “Protocol” and the “Transport” on the client and server for thrift RPC?

Answer 13

Protocol ensures that the data encoding formatting matches for the server and the client (JSON or otherwise)

Transport ensures that the network protocols that the server and client use are consistent

Question 14

In what way does a thrift client violate the principle of location transparency?

Answer 14

The thrift client must know the host name and port number of a given server

Question 15

In what way does a thrift client-server system violate the principle of location transparency?

Answer 15

Server may throw a variety of exceptions, and the exceptions will be passed along the network to the Client

Question 16

Thrift _______, ________ and ______ _______ are not thread safe in a multi-threaded client

Answer 16

transports
protocols
client stubs

Question 17

Asynchronous thrift RPC clients must provide a _______ ______ to be invoked upon completing a request

Answer 17

callback function

Question 18

Asynchronous thrift RPC clients must use a non-blocking ________ and ________

Answer 18

socket

transport

Question 19

What are some synchronization primitives used between a callback method on an asynchronous RPC client, and an RPC server (in java)?

Answer 19

Conditions
or
CountDownLatch

Question 20

What steps must an Asynchronous thrift RPC client program execute when calling a service handler and invoking an RPC from the server?

Answer 20

Client must:
1. Create a TNonblockingTransport object using the network address of the server (internally creates a socket connection)

2. Create a TProtocolFactory object
3. Create an TAsyncClientManager object
4. Access the server async RPC object from the objects created in steps 1, 2, and 3

Question 21

What steps must a multi-threaded thrift RPC server program execute when setting up a server which allows clients to invoke RPC calls to it?

Answer 21

Server must:
1. Create the server socket object with a port number to bind to

2. Create objects for the ProtocolFactory, TransportFactory, and the ProcessorFactory. Pass them into the server object
3. Create the server object from the objects in step 2, and pick the appropriate server type (HsHa, ThreadPool, etc)
4. Set the max number of worker threads
5. run the blocking “serve()” call

Question 22

What are the names of the 5 thrift server implementations in Java?

Answer 22

• TSimpleServer
• TNonblockingServer
• THsHaServer
• TThreadedSelectorServer
• TThreadPoolServer

Question 23

How does the thrift TSimpleServer in Java work?

Answer 23

TSimpleServer uses a single thread and blocking I/O

Question 24

How does the thrift TNonblockingServer in Java work?

Answer 24

TNonblockingServer uses a single thread and non blocking I/O. It can handle parallel connections but executes requests serially just like TSimpleServer.

Question 25

How does the thrift THsHaServer in Java work?

Answer 25

THsHaServer uses one thread for network I/O and a pool of worker threads. It can process multiple requests in parallel.

Question 26

How does the thrift TThreadedSelectorServer in Java work?

Answer 26

TThreadedSelectorServer uses a pool of threads for network I/O and a pool of worker threads for request processing

Question 27

How does the thrift TThreadPoolServer in Java work?

Answer 27

TThreadPoolServer uses one thread to accept connections and then handles each connection using a dedicated thread drawn from a pool of worker threads

Question 28

When creating a multi-threaded thrift server in Java, where how does the following error: "OutOfMemoryError: Unable to create new native thread" happen?

Answer 28

When a Java program attempts to create a large number of threads (eg 100+)

Question 29

Why does TThreadedPoolServer yield the best performance internally? What is the drawback with this type of server?

Answer 29

- Performance is maximized because it uses minimal synchronization internally - Drawback is that it consumes 1 thread per connection leading to resource exhaustion when there are many parallel connections

Question 30

What should be the maximum number of size of the worker thread pool when creating a multi-threaded server in java?

Answer 30

64 threads This avoids resource exhaustion

Question 31

What is the purpose of the thrift middleware layer protocols (on top of the transport layer)?

Answer 31

The middleware layer protocols are used to encode RPC requests and responses (such as raw text, JSON, XML, etc)

Question 32

What are 3 thrift middleware protocol types?

Answer 32

- TBinaryProtocol - TCompactProtocol - TJSONProtocol

Question 33

What is the difference between TBinaryProtocol and TCompactProtocol?

Answer 33

BinaryProtocol encodes numeric values in a straightforward binary format whereas CompactProtocol is more compact and uses variable-length encoding for integers

Question 34

Thrift fields can be modified in a manner that provides compatibility between old and new protocol versions, under 4 rules. What are these 4 rules?

Answer 34

1. The manually assigned numeric tags of existing fields should never be changed 2. New fields can be added as long as they are declared "optional" and have appropriate default values 3. Fields that are no longer needed can be removed as long as their tag numbers are never reused. Unrecognized fields are skipped by the parser, but old servers may be unable to process requests from new clients 4. Default values can be changed