Comet (programming)

Template:Articleissues

Template:Otheruses2 In web development, Comet is a neologism to describe an web application model in which a long-held HTTP connection allows a web server to push data to a browser, without the browser explicitly requesting it. Comet is an umbrella term that uses multiple techniques for achieving this interaction. All methods have in common that they rely on browser-native technologies such as JavaScript, rather than on proprietary plugins. In theory, the Comet approach differs from the original model of the web, in which a browser requests a complete web page or chunks of data to update a web page. However in practice, Comet applications typically use Ajax with long polling to detect new information on the server. The concept has been known by various names, including Ajax Push, Reverse Ajax, Two-way-web, HTTP Streaming and HTTP server push among others.

Traditional web architectures

Web applications have historically been less flexible and capable than desktop applications counterparts, partly because of limitations of creating dynamic user interfaces and constraints in the online environment. One limitation that Comet seeks to address is the inability of a web server to choose when to send updated information to a web browser.

Page-by-page model

Main article: World Wide Web

Traditionally, each chunk of information sent by a web server to a browser must be explicitly requested. Each time the user opens a new web page, the browser initiates an HTTP connection to the web server. The browser requests a page for which the server returns the HTML code and waits for more requests on the same connection. The browser typically makes multiple HTTP requests, for each resource used by the page the server responds with data, e.g. images and stylesheets. For example Wikipedia is a web application built using this page-by-page web model.

Ajax with polling

Main article: Ajax (programming)

To overcome limitations of the page-by-page model, some web applications use Ajax (asynchronous JavaScript and XML). With Ajax the browser uses JavaScript to modify the current page rather than creating a new page for each request. This limits the server’s response to only the relevant information. Since multiple Ajax request can be handled at the same time, users can interact with a page even while data is retrieved. Some web applications regularly poll the server to ask if new information is available. High polling frequencies can waste server resources and bandwidth, however web applications with relatively infrequent updates can use polling without significant overhead.

Web server push

In 1995, Netscape Navigator introduced “server push” technology, allowing web applications to update images or HTML pages at will, by sending new versions of a resource as separate parts of a multipart HTTP response (at the same time, Netscape also introduced “client pull”, a way for Navigator to re-request a page after a specific amount of time, analogous to the ajax with polling model described above). Other web browsers implemented this feature, such as Mozilla Firefox.

Implementation of Comet

Template:Expert-subject

Comet is an umbrella term for technologies that attempt to eliminate both the limitations of the page-by-page model and traditional polling. Comet-like applications offer real-time interaction by relying on a persistent HTTP connection (or where not possible a long lasting HTTP connection) to provide the browser with updates as designated by the web application.

Unfortunately, browsers and proxies are not designed with server events in mind. Web application developers have tried to work around several unintended side-effects to implement Comet-like behavior, each with different benefits and drawbacks. In particular, the HTTP 1.1 specification states that a browser should not have more than 2 simultaneous connections with a web server. However, holding one connection open for real-time events has an negative impact on browser usability. The browser may be blocked from sending a new request while it still loads, for example a series of images. This can be worked around by creating a distinct hostname for real-time information, which be hosted on the same physical server.

Specific methods of implementing of Comet fall into two major categories: streaming and long polling.

Streaming

In an application using streaming Comet, the browser opens a single persistent connection to the server for all Comet events, which is handled incrementally on the browser side. Each time the server sends a new event, the browser interprets it, but neither side closes the connection. Specific technologies for accomplishing streaming Comet include the following.

Hidden IFrame

A basic technique for dynamic web application is to use a hidden IFrame HTML element (an inline frame, which allows a website to embed one HTML document inside another). This invisible IFrame is sent as a chunked block, which implicitly declares it as infinitely long (sometimes called “forever frame”). As events occur, the iframe is gradually filled with script tags, containing JavaScript to be executed in the browser. Because browsers render HTML pages incrementally, each script tag is executed as it is received.

One benefit of the IFrame method is that it works in every common browser. Two downsides of this technique are the lack of a reliable error handling method, and the impossibility of tracking the state of the request calling process.

XMLHttpRequest

The XMLHttpRequest (XHR) object, the main tool used by Ajax applications for browser–server communication, can also be pressed into service for server–browser Comet messaging, in a few different ways.

In 1995, Netscape Navigator added a feature called “server push”, which allowed servers to send new versions of an image or HTML page to that browser, as part of a multipart HTTP response (see History section, below), using the content type multipart/x-mixed-replace. Since 2004, Gecko-based browsers such as Firefox accept multipart responses to XHR, which can therefore be used as a streaming Comet transport. On the server side, each message is encoded as a separate portion of the multipart response, and on the client, the callback function provided to the XHR onreadystatechange function will be called as each message arrives. This functionality is only included in Gecko-based browsers, though there is discussion of adding it to Webkit. Additionally, it is currently impossible to determine when the multipart XHR connection has been closed.

Instead of creating a multipart response, and depending on the browser to transparently parse each event, it is also possible to generate a custom data format for an XHR response, and parse out each event using browser-side JavaScript, relying only on the browser firing the onreadystatechange callback each time it receives new data.

Ajax with long polling

None of the above streaming transports works across all modern browsers without causing negative side-effects in any—forcing Comet developers to implement several complex streaming transports, switching between them depending on the browser. Consequently many Comet applications instead opt for long polling, which is easier to implement on the browser side, and works, at minimum, in every browser that supports XHR. As the name suggests, long polling requires the client to poll the server for an event (or set of events). The browser makes an Ajax-style request to the server, which is kept open until the server has new data to send to the browser, which is sent to the browser in a complete response. The browser initiates new long polling request in order to obtain subsequent events.

Unlike with polling, long polling sends an event payload along with most poll responses. While in the worst case—frequently occurring events with small payloads—long polling incurs significant computational overheadTemplate:Fact, Greg Wilkins found that in all cases, long polling either equals or outperforms polling. Streaming has better performance still, as it eliminates much of the per-message overheadTemplate:Fact.

Specific technologies for accomplishing long-polling include the following.

XMLHttpRequest long polling

For the most part, XMLHttpRequest long polling work like any standard use of XHR. The browser makes an asynchronous request of the server, and provides a function to be called when the server has responded. At the end of this callback function, the browser creates and sends another XHR, to await the next event. Thus the browser always keeps a request outstanding with the server, to be answered as each event occurs.

Script tag long polling

While any Comet transport can be made to work across subdomains, none of the above transports can be used across different second-level domains (SLDs), due to browser security policies designed to prevent cross-site scripting attacks. That is, if the main web page is served from one SLD, and the Comet server is located at another SLD, Comet events cannot be used to modify the HTML and DOM of the main page, using those transports. This can be worked around by creating a proxy server in front of one or both sources, making them appear to originate from the same domain; however, this is often undesirable, for complexity or performance reasons.

Unlike IFrames or XMLHttpRequest objects, script tags can be pointed at any URI, and JavaScript code in the response will be executed in the current HTML document. This creates a potential security risk for both servers involved, and though the risk to the data provider (in our case, the Comet server) can be avoided using “JSONP”. As Bob Ippolito explains, “Your page is still toast if the remote host decides to inject malicious code instead of JSON data”.

A long-polling Comet transport can be created by dynamically creating script elements, and setting their source to the location of the Comet server, which then sends back JavaScript (or JSONP) with some event as its payload. Each time the script request is completed, the browser opens a new one, just as in the XHR long polling case. This method is the only cross-browser choice for developers looking to implement cross-domain Comet.

Problems

Because Comet applications send events in real time, they typically use more resources than other types of web applications, making them more difficult to scale (grow to support large numbers of users). Template:Fact

Proxy servers and firewalls between the browser and web server can pose network problems. Firewalls are often configured to drop connections that have been idle for too long, applications need to tear down and recreate the Comet connection periodically.Template:Fact Transparent HTTP proxies may buffer data in chunks up to 32 or 64 kilobytes which prevents streaming connections, applications need to detect when non-polling HTTP communication is impossible.Template:Fact

History

Template:Copyedit

Early Java applets

The ability to embed Java applets into browsers opened richer real-time options. An applet can open a raw TCP socket to the server from which it has loaded. This socket can remain open as long as the browser keeps open the document which hosts the applet. Event notifications can be sent in any format—text or binary—and decoded by Java code in the applet.

Despite their capabilities, however, Java applets never really took off. User interfaces in Java applets were often buggy and frustrating for users, suffering graphical glitches. Communications on ports other than port 80 are blocked by aggressive firewalls. Not all users had the proper version of the Java plugin installed in their browsers, or even any version at all. When possible, web application developers preferred to use browser-native technologies (see Disadvantages of Java applets). To mitigate these disadvantages, some developers such as Caplin Systems, used Java applets as a transport mechanism, leaving application-specific interface logic to browser-native JavaScript.

Early push applications

Circa 2000, developers began creating the first push applications using client-side JavaScript inside a Frame/IFrame. Pushlets, a framework created by the Dutch Just van den Broecke, was one of the first open source implementations, based on server-side Java servlets, and a client-side JavaScript library. The 2000 startup KnowNow built several push demonstrations, and planned to create a platform on top of the Web, which would be used by other developers to implement real-time web apps. After releasing their mod_pubsub server prototype, KnowNow turned to enterprise applications.Template:Fact Bang Networks—a 1999 Silicon Valley start-up backed by Netscape co-founder Marc Andreessen and lavishly financed—attempted to create a real-time push standard for the whole Web, but ran out of cash and folded at the end of 2003.Template:Fact Lightstreamer, an Italian company, focused on the financial market with their Java-based server.Template:Fact

First Comet applications

In March 2006, software engineer Alex Russell coined the term Comet in a post on his personal blog. The new term was a play on Ajax (Ajax and Comet both being common household cleansers). This umbrella term for previously existing concepts gained currency as a moniker, and began appearing at web-related technology conferences.

In 2006, some applications exposed Comet to a wider audience: Meebo’s multi-protocol web-based chat application enables users to connect to AOL, Yahoo, and Microsoft chat platforms through the browser; Google added web-based chat to Gmail; JotSpot, a startup since acquired by Google, built Comet-based real-time collaborative document editing; Comet-based chat is the centerpiece of the event-planning site Renkoo. Meanwhile, Comet continued to make inroads into enterprise markets. For instance, new enterprise Comet companies and solutions were created such as the Java-based ICEfaces JSF framework. Others that had previously used Java-applet based transports, such as Caplin Systems, switched instead to pure-JavaScript implementations.

Alternatives

Browser-native technologies are inherent in the term Comet. Attempts to improve non-polling HTTP communication have come from multiple sides:

• The HTML 5 draft specification produced by the Web Hypertext Application Technology Working Group (WHATWG) specifies so called server-sent events, it offers a new HTML element, event-source and a new data format called DOM event stream.
• The Bayeux protocol by the Dojo foundation. It leaves browser-specific transports in place, and defines a higher-level protocol for communication between browser and server, with the aim of allowing re-use of client-side JavaScript code with multiple Comet servers, and allowing the same Comet server to communicate with multiple client-side JavaScript implementations. Bayeux is based on a publish/subscribe model, so servers supporting Bayeux have publish/subscribe built-in.
• The BOSH protocol by the XMPP standards foundation. It emulates a bidirectional stream between browser and server by using multiple synchronous HTTP connections.
• The JSONRequest object, proposed by Douglas Crockford, would be an alternative to the XHR object, with support for Comet.
• Use of plugins, such as Java applets or the proprietary Adobe Flash (Java BlazeDS is a server plugin which streams events to Flash applications). These have the advantage of working identically across all browsers with the appropriate plugin installed, need not rely on HTTP connections, and face none of the security restrictions placed on browser-native transportsTemplate:Clarifyme.

See also

• Push technology
• Pull technology
• Web 3.0

External links

• Comet Daily – a website dedicated to articles on Comet techniques
• HTTP Streaming at Ajax Patterns

es:Comet hu:Comet mk:Comet ja:Comet pt:Comet (programação)