Twisted (software)

Twisted

Original author(s)	Glyph Lefkowitz
Developer(s)	Community
Initial release	October 22, 2002; 22 years ago (2002-10-22)[1]
Stable release	24.11.0[2]  / 2 December 2024; 30 days ago (2 December 2024)
Repository	github.com/twisted/twisted
Written in	Python
Type	Event-driven networking
License	MIT License
Website	www.twistedmatrix.com

Twisted is an event-driven network programming framework written in Python and licensed under the MIT License.

Twisted projects variously support TCP, UDP, SSL/TLS, IP multicast, Unix domain sockets, many protocols (including HTTP, XMPP, NNTP, IMAP, SSH, IRC, FTP, and others), and much more. Twisted is based on the event-driven programming paradigm, which means that users of Twisted write short callbacks which are called by the framework.

Core ideas

Separation of protocols and transports

Twisted is designed for complete separation between logical protocols (usually relying on stream-based connection semantics, such as HTTP or POP3) and transport layers supporting such stream-based semantics (such as files, sockets or SSL libraries). Connection between a logical protocol and a transport layer happens at the last possible moment — just before information is passed into the logical protocol instance. The logical protocol is informed of the transport layer instance, and can use it to send messages back and to check for the peer's identity. Note that it is still possible, in protocol code, to deeply query the transport layer on transport issues (such as checking a client-side SSL certificate). Naturally, such protocol code will fail (raise an exception) if the transport layer does not support such semantics.

Deferreds

Central to the Twisted application model is the concept of a deferred (elsewhere called a future). A deferred is an instance of a class designed to receive and process a result which has not been computed yet, for example because it is based on data from a remote peer. Deferreds can be passed around, just like regular objects, but cannot be asked for their value. Each deferred supports a callback chain. When the deferred gets the value, it is passed to the functions on the callback chain, with the result of each callback becoming the input for the next. Deferreds make it possible to operate on the result of a function call before its value has become available.

For example, if a deferred returns a string from a remote peer containing an IP address in quad format, a callback can be attached to translate it into a 32-bit number. Any user of the deferred can now treat it as a deferred returning a 32-bit number. This, and the related ability to define "errbacks" (callbacks which are called as error handlers), allows code to specify in advance what to do when an asynchronous event occurs, without stopping to wait for the event. In non-event-driven systems, for example using threads, the operating system incurs premature and additional overhead organizing threads each time a blocking call is made.

Thread support

Twisted supports an abstraction over raw threads — using a thread as a deferred source. Thus, a deferred is returned immediately, which will receive a value when the thread finishes. Callbacks can be attached which will run in the main thread, thus alleviating the need for complex locking solutions. A prime example of such usage, which comes from Twisted's support libraries, is using this model to call into databases. The database call itself happens on a foreign thread, but the analysis of the result happens in the main thread.

Foreign loop support

Twisted can integrate with foreign event loops, such as those of GTK+, Qt and Cocoa (through PyObjC). This allows using Twisted as the network layer in graphical user interface (GUI) programs, using all of its libraries without adding a thread-per-socket overhead, as using Python's native library would. A full-fledged web server can be integrated in-process with a GUI program using this model, for example.

Applications using Twisted

• The BuildBot continuous-integration system relies on Twisted for client/server communication.^[3]
• ITA Software has developed an airline-reservation system for Air Canada that uses Twisted extensively.^[4]
• SageMath, an open-source alternative to Mathematica, Maple, Magma, MATLAB, has a web-based interface, SageMath notebook,^[5] that runs on a Twisted server.^[6]
• Twisted was used in the Omegle one-on-one chat service^[7] until it was replaced with gevent for performance reasons.^[8]
• The Apple Calendar Server uses Twisted,^[9] as do some internal projects of NASA.
• Conch, an implementation of the Secure Shell (SSH) protocol
• The original version of social networking and microblogging site Jaiku used Twisted.^{[citation needed]}
• Fluidinfo, an online cloud data-store, uses Twisted extensively for internal RPC (partly in combination with Thrift and AMQP), for its internal services, and for external APIs.
• The file-hosting service Ubuntu One used Twisted.
• Tor2web, an HTTP proxy for Tor Hidden Services (HS), uses Twisted.
• GlobaLeaks, an open-source whistleblowing framework, uses Twisted.
• Cloudkick, a cloud-server management web-application, used Twisted. It now has been rewritten using Node.js.
• Twilio, a cloud telephony provider, uses Twisted.
• Twitch, a video game broadcasting and chat community, uses Twisted.^[10]
• Velocity Weather, a meteorological data processing and integration API is built on Twisted.^[11]
• qwebirc, a web-based IRC client, uses Twisted.
• Zenoss Core, a network management platform, uses Twisted for many internal and collection daemons.
• Scrapy, a web crawler based on Twisted.
• Listen to Wikipedia, a Wikipedia audio-visualizer, uses Twisted to broadcast real-time edit events to browsers.^[12]
• Tahoe-LAFS, a distributed data store and distributed file system.
• Deluge, a highly modular BitTorrent client, uses Twisted.^[13]
• Magic Wormhole, a secure file transfer tool using PAKE.^[14]

Nevow

Original author(s)	Donovan Preston[15][16][17]
Developer(s)	Divmod
Initial release	May 6, 2004; 20 years ago (2004-05-06)
Stable release	0.14.4 / June 14, 2018; 6 years ago (2018-06-14)[18]
Repository	github.com/twisted/twisted
Written in	Python
Operating system	Cross-platform
Type	Web application framework
License	MIT License
Website	www.twistedmatrix.com

Nevow (pronounced like the French nouveau) is a Python web application framework originally developed by the company Divmod. Template substitution is achieved via a small Tag Attribute Language, which is usually embedded in on-disk XML templates, though there is also a pure-Python domain-specific language called Stan, for expressing this markup programmatically. Nevow integrates well with Twisted.

Nevow was deployed on several high-profile web sites, most notably the official Python site.^[19]

As of mid-2010, Divmod went out of business,^[20] causing development work on Nevow to all but cease, and in 2011 its homepage was no longer accessible.^[21] There is a project on Launchpad, hosting the source code of Divmod^[22] including the source code of the Nevow project.^[23]

Athena

Athena is a Nevow component which facilitates bi-directional, asynchronous communication between the Python and JavaScript portions of a web application in the form of remote procedure calls. This technique is typically called Ajax or Comet, though Nevow's implementation predates both of these labels.^[24] Athena also includes an inheritance-based JavaScript object system, which forms the basis of a client-side widget abstraction, module system and in-browser unit testing kit.

See also

• Application server
• Reactor pattern
• Perl Object Environment, a comparable framework for the Perl programming language
• Netty, for the Java programming language
• Node.js, for Javascript
• EventMachine, an event-processing library for Ruby^[25]
• Kivy (framework), a multi-platform GUI framework (including iOS and Android)^[26]

References

1. Shtull-Trauring, Itamar (2002-10-22). "Twisted 1.0". twisted-python (Mailing list). Retrieved 2008-08-14.
2. "Release 24.11.0". 2 December 2024. Retrieved 28 December 2024.
3. "BuildBot Manual". github.com. Archived from the original on 2012-07-29. Retrieved 2017-10-28.
4. "Python Slithers into Systems". eweek.com.
5. SageMath#Features
6. Sage a Basic Overview^{[permanent dead link‍]}
7. "mistah deejay". omegler.blogspot.com.
8. "Google Groups". groups.google.com.
9. "Calendar and Contacts Server". www.calendarserver.org.
10. "Twitch - Jobs". Twitch.
11. "Velocity Weather -". www.velocityweather.com.
12. A WebSocket-oriented monitor for Wikipedia (also, wikimon, wikital monsters), hatnote, 2017-09-03, retrieved 2017-09-21
13. "Deluge: build from source". dev.deluge-torrent.org. Retrieved 2020-01-08.
14. "Magic Wormhole". github.com.
15. this 2004 paper by Donovan Preston, the original author of Nevow and Formless Archive index at the Wayback Machine
16. Nevow: A Web Application Construction Kit, Donovan Preston
17. Subject: Re: ANN: Twisted 1.1.0, the framework of *your* internet, From: Donovan Preston, Date: 2003-10-27, Newsgroups: gmane.comp.python.twisted, nevow was originally a research project -- a weekend's worth of hacking whose benefits turned out to be so great that we decided to focus on bringing it up to a usable state for Quotient as soon as possible. And that's where it is now.
18. "Releases · twisted/nevow". GitHub. Retrieved 2020-11-04.
19. PythonWebsitePyramidDocs, Pyramid is the internal website templating system used to build the python.org pages. (It is not connected to the Pyramid web framework, formerly named BFG, that's used with Pylons.) Pyramid uses Nevow as its templating system and YAML as the format for its file based data - PythonInfo Wiki, python.org
20. Goodbye, Divmod. Hello, World!, July 10, 2009, Deciphering Glyph
21. Divmod Nevow commits Archived 2012-10-15 at the Wayback Machine on Ohloh
22. Divmod Archived August 13, 2006, at the Wayback Machine
23. Divmod on Launchpad
24. History Archived 2016-03-03 at the Wayback Machine, Nevow Athena from Scratch, or The Evolution of a Chat Application — Divmod Nevow vtrunk documentation
25. "EventMachine". Retrieved 20 August 2011.
26. "Integrating with other Frameworks — Kivy 1.11.0.dev0 documentation". kivy.org.

External links

• Official website