Patent US10270817

Forward request queuing in a distributed edge processing environment
[+] Show Details

US10270817

1. Apparatus for use in a distributed network, comprising: (3) (3)

at least one hardware processor, an operating system, and a process executed by the hardware processor that manages incoming client requests and selectively forwards given service requests to at least one other server in the distributed network;

storage coupled to the processor and including a set of queues, wherein each queue in the set of queues is unique associated with a particular hostname; and

code tangibly embodied on computer media and executed by the hardware processor, the code comprising: first code operative to make a first determination that a request associated with the particular hostname should be sent to the other server in the distributed network, and conditioned upon the first determination being made, to make a second determination whether the request should be sent to the queue associated with the particular hostname rather than sent forward to the other server, thereby overriding the first determination, wherein the second determination regarding whether the request should be sent to the queue associated with the particular hostname rather than sent forward to the other server is based on presence in the queue of at least one other request associated with the particular hostname and one of: a value of an estimated load on the other server, and a value representing a maximum number of allowed concurrent connections to the other server;

second code operative upon determining that the request should be sent forward to the other server to modify the estimated load value and initiate forwarding of the request;

third code operative upon determining that the request should be sent to the queue to insert the request into the queue based on one of: a priority, and an arrival time.

2. The apparatus as described in claim 1 wherein the particular hostname is a name associated with a content delivery network (CDN) customer. (1) (0)

3. The apparatus as described in claim 2, wherein the process is controlled by metadata uniquely associated with the particular hostname. (4) (0)

4. The apparatus as described in claim 3 wherein the metadata identifies a maximum number of requests associated with the particular hostname that may be aggregated in the queue associated with the particular hostname. (0)

5. The apparatus as described in claim 3 wherein the metadata identifies a maximum time that a given request associated with the particular hostname may remain in the queue associated with the particular hostname. (0)

6. The apparatus as described in claim 3 wherein the metadata identifies a number of given requests associated with the particular hostname that may be released from the queue associated with the particular hostname at a given time. (0)

7. The apparatus as described in claim 3 wherein the first code aggregates in the queue associated with the particular hostname one or more requests associated with the particular hostname when a rate of incoming client requests associated with the particular hostname exceeds a limit. (2) (0)

10. The apparatus as described in claim 1, further including fourth code responsive to a third determination that a maximum number of requests associated with the particular hostname are in the queue associated with the particular hostname for taking a given action. (2) (0)

13. The apparatus as described in claim 1, wherein the code adjusts the estimated load and thus the rate at which requests associated with the particular hostname are forwarded to the other server by a number of failures or slow connections encountered under a current estimated load. (0)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Patent US10270817

Forward request queuing in a distributed edge processing environment

Patent Matrix® Search