Talk:Parallel computing/Archive 1

This is an archive of past discussions about Parallel computing. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 1

UHHH!!!! This page is really gross and out of date :(

Very very slightly better now. The first section still needs further breakup and generalisation and there should be more in the pre-toc sumary part. Also, the software part needs expansion and something should be done with the 'general topics' part.

I managed to refine it. It is better now. :)--Leo 03:32, 13 April 2006 (UTC)

These two are under programming languages but they aren't, they are library for C/C++ and FORTRAN High level programming languages.

Strictly, OpenMP is a language extension API while MPI is a libary API. But informally, they are called programming languages by many users. --Leo 22:24, 12 April 2006 (UTC)

If there's anybody who actively maintains this article, I think it ought to be rephrased a bit to at least acknowledge that TLP and multiprocessing aren't the only forms of parallel computing; ILP just as significant. -- uberpenguin 07:41, 11 December 2005 (UTC)

I'm proposing that Parallel programming be merged into Parallel computing, since the bulk of the content (what little there is) in Parallel programming is already contained in the Parallel computing article (which also provides more context). Is there significant additional content that can be added to Parallel programming to justify keeping it a separate article? --Allan McInnes 20:37, 13 January 2006 (UTC)

Nope.. That article can easily be included as a subheading of this one. Go ahead and merge them. -- uberpenguin 21:16, 13 January 2006 (UTC)

I have put up a proposal for a concurrency wikiproject at User:Allan McInnes/Concurrency project. Input from all interested parties (especially those who will actually contribute) is most welcome. --Allan McInnes 21:31, 20 January 2006 (UTC)

Update: the concurrency wikiproject has been subsumed by the larger (and more active) WikiProject Computer science. --Allan McInnes (talk) 17:10, 3 April 2006 (UTC)

What is the status of the merge? —Preceding unsigned comment added by Adamstevenson (talk • contribs)

The merge was completed on January 20, 2006. Parallel programming is now a redirect to Parallel computing. --Allan McInnes (talk) 17:10, 3 April 2006 (UTC)

OpenMP is great, but if you go to the official website, you realize the last posting of "In the Media" is 1998. You then realize this is for Fortran and C++.

My point is the world has moved on to multicore chips in $900 laptops, Java and C#. Parallel computing has moved _much_ farther than OpenMP has. We need more content here for the masses, not the PhD's on supercomputers.

I tried to post _recent_ information about new parallel frameworks like DataRush, but was 'smacked down' by Eagle_101 (or some close facsimile) and all my content was obliterated.

QUESTION: Why is it okay for Informatica, Tibco and other large corporations to post massive marketing datasheets in wikipedia, but not allow subject matter experts to post useful information in appropriate topic areas?

I concede DataRush is free but not open source, nor a global standard. But indeed it is recent, topical and factual. The programming framework exists, is useful to mankind etc...etc... So why does wikipedia not want to inform the world of its existence? —Preceding unsigned comment added by EmilioB (talk • contribs)

Well, you might start by contributin actual content, instead of just a link. The fact that the link looks a lot like an attempt to advertise (which is generally discouraged on Wikipedia) probably didn't help matters. --Allan McInnes (talk) 04:19, 1 December 2006 (UTC)

There are two key frameworks in the Java community -- DataRush and Javolution.

Both provide "hyper-parallel" execution of code when you use the frameworks.

http://www.pervasivedatarush.com

http://www.javolution.org

Emilio 16:39, 3 January 2007 (UTC)EmilioB

"One woman can have a baby in nine months, but nine women can't have a baby in one month." - Does anyone know the origin of this quote? Raul654 17:57, 3 April 2007 (UTC)

I first heard it posed as an accepted belief at NASA Goddard Spaceflight Center, but I don't know who originated it. Howard C. Berkowitz 01:52, 4 July 2007 (UTC)

Yes, I've also heard it attributed to various people at NASA. But apparently the quote came from IBM:

According to the Brooks' law, Fred Brooks said "The bearing of a child takes nine months, no matter how many women are assigned." --75.39.248.28 07:01, 25 August 2007 (UTC)

AFAIK, a modern processor's power is linear with clock speed, not "super linear". This section could also probably do with some references to back up any claims.

I frankly can't believe nobody mentioned the first computer to use parallell processing, & the year. What was it? Trekphiler 17:36, 25 August 2007 (UTC)

Off the top of my head, ILLIAC II? Raul654 01:50, 31 August 2007 (UTC)

Good guess. ILLIAC IV. Thanks. (So howcum it isn't in the article...?) Chief O'Brien 04:49, 3 September 2007 (UTC)

One statement of conditions necessary for valid parallel computing is:

   * I_j \cap O_i = \varnothing, \,
   * I_i \cap O_j = \varnothing, \,
   * O_i \cap O_j = \varnothing. \,

Violation of the first condition introduces a flow dependency, corresponding to the first statement producing a result used by the second statement. The second condition represents an anti-dependency, when the first statement overwrites a variable needed by the second expression. The third and final condition, q, is an output dependency.

The variable q is referenced but not defined. The third condition is a copy of the second. MichaelWattam (talk) 17:24, 18 March 2009 (UTC)

Quite right, the text for the third condition made no sense. That error was mine, I'm a bit embarrassed I haven't seen it earlier. q was a typo I introduced when I first wrote that part back in 2007. No, the third condition is separate from the second: hopefully it makes more sense now. henrik•talk 19:42, 18 March 2009 (UTC)

Is this statement in the article referring to interlocks?

Only one instruction may execute at a time—after that instruction is finished, the next is  executed

--Ramu50 (talk) 03:13, 15 June 2008 (UTC)

What is an interlock? I've never heard this term used in any parallel computing sense. Raul654 (talk) 03:19, 15 June 2008 (UTC)

Read the Wikipedia Article "MAJC" paragraph 5, and you will understand it. It is sort of a method of rendering instructions codes in CPU. --Ramu50 (talk) 22:55, 15 June 2008 (UTC)

The article you link to (which, frankly, is not well written at all) defines an interlock as "pauses in execution while the results of one instruction need to be processesed for the next to be able to run". I don't know where the term interlock came from, but in a pipelined processor the situation described would cause a data hazard (specifically, a data hazard of the read after write variety). This can be resolved either by bubbling (inserting No-ops into the pipeline at the point of contention) or forwarding from a later processor stage to an earlier one.

And to answer your earlier question, the above statement (Only one instruction may execute at a time—after that instruction is finished, the next is executed) refers to a serial (that is, non-pipelined) processor. The "interlock" (data hazard) you describe cannot occur on such a processor. Raul654 (talk) 02:08, 16 June 2008 (UTC)

This article is really bad - even the first sentence has a non-trivial error ("Parallel computing is the simultaneous execution of some combination of multiple instances of programmed instructions and data on multiple processors in order to obtain results faster" -- this totally ignores ILP). I'm going to rewrite it. I don't have my copies of Patterson and Hennessey handy - they're in my office. I'll pick them up from the office tomorrow. In the meantime, I've started the rewrite at User:Raul654/PC Raul654 04:11, 28 October 2007 (UTC)

Note - I'm about halfway finished. Raul654 04:31, 6 November 2007 (UTC)

Raul - this is a nice, well illustrated article. While I can understand your redirecting Concurrent computing to here, I think you are being a tad overzealous in redirecting Distributed computing to this page. You've consigned the entire subject under the rather obscure-sounding subheading of "Distributed memory multiprocessing". Suddenly, Wikipedia seems to have lost the rather important subject of Distributed computing! Please reconsider your action and undo the redirect on Distributed computing. That page may still need more work, but it's way too big a sub-topic of parallel computing to cram into this page. - JCLately 06:18, 7 November 2007 (UTC)

Actually, I was very careful to define "distributed computing" in the very first paragraph of the article as computing across computers connected by a network (and cite a reference for that definition) Under the classes I have laid out here, that would be distributed memory multiprocessing (which includes clusters and MPPs) and grid computing - which is a *very* accurate definition. At the same time, I think you are exaggerating (greatly) the distinction between parallel computing and distributed computing. Raul654 06:23, 7 November 2007 (UTC)

I'm too tired to do this justice right now, but I'll at least make one brief reply before I hit the sack. I won't argue with your formal classification of the subject area, but the term "parallel computing" generally connotes broader and more fundamental aspects of parallelism that come up in the design of processors and supercomputers. Distributed computing may be conceptually equivalent, but practically speaking, the introduction of significant distances and greater autonomy in distributed systems is a big deal. It brings in issues of networking, databases, and the wonderful world of middleware. This is too much to cover in under "Parallel computing", and I'd venture to guess that most people who are interested in distributed computing - a very hot topic - would not be so interested to read about the generalities and fundamental issues covered nicely on this page. I'm guessing that you have a somewhat academic perspective on this subject, which is not universally shared - that may be an understatement. - JCLately 06:49, 7 November 2007 (UTC)

After some thinking, I've reworked that section: I've renamed distributed memory multiprocessing to distributed computing (the two are exactly synonomous) and put grid computing under that heading. I don't consider database/network/middleware discussion to warrant a separate distributed computing article (those topics should more properly be discussed in the specific articles like cluster and grid computing). But if you want distributed-computing specific discusion, there is space under that heading now. Raul654 16:59, 7 November 2007 (UTC)

I agree that renaming the section from "distributed memory multiprocessing" to "distributed computing" is an improvement, but I do not share your view that the general topic of distributed computing is unworthy of its own page. Consider the fact that a Google search on the phrase "distributed computing" yields 1.7 million hits, which happens to be somewhat more that the number of hits for the phrase "parallel computing". Also note that Wikipedia's "What links here" lists over 500 other WP pages that link to Distributed computing, and the number of WP links to Parallel computing is somewhat less. Even if we set aside the rather substantial editing implications of the redirect you proposed, is there a good reason to presume that all of those links to Distributed computing really should have pointed to the broader topic of Parallel computing? Certainly, these terms are not synonymous.

The analogy that comes to mind is the relationship between physics and chemistry. From the physicist's point of view, chemistry is just the physics of tightly bound configurations of fermions. And your acknowledgment of grid and cluster computing as legitimate topics is akin to recognizing the subfields of organic and inorganic chemistry, but being for some reason unwilling to acknowledge the broader field of chemistry. That's about as far as I can go with that analogy, because distributed computing doesn't break down quite so neatly: the concept of grid computing is rather fuzzy, and might be regarded by some as a buzzword. Cluster computing isn't quite so fuzzy, but there is a great deal to be said about distributed computing that doesn't specifically belong under either cluster or grid computing. At least the topic of distributed computing has the virtue of being widely recognized as meaningful. (Incidentally, I don't think "Massive parallel processing" in the sense of MPP supercomputer architectures really belongs under Distributed computing, but that's a different issue.)

From your perspective, the database/network/middleware aspects may be of only passing interest, but to many people interested in distributed computing those considerations - and let me add security to the list - these are issues of paramount interest. Returning to my analogy, someone interested in chemistry is not looking for a page that discusses quarks and gluons. Given that there are a number of subtopics of distributed computing that are worthy of their own separate pages, and there also exists an extensive category tree for Category:Distributed computing, can there be any reasonable doubt that Distributed computing is a topic worthy of its own page? - JCLately 04:31, 8 November 2007 (UTC)

Hi, below is a suggestion for a new introductory paragraph. Comments? henrik•talk 06:51, 9 November 2007 (UTC)

Parallel computing is a form of computing in which many operations are carried out simultaneously. Parallel computing operates on the principle that large problems can almost always be divided into smaller ones, which may be carried out concurrently ("in parallel"). Parallel computing has been used for many years, mainly in high performance computing, but interest has been renewed in later years due to physical constraints preventing frequency scaling. Parallel computing has recently become the dominant paradigm in computer architecture, mainly in the form of multicore processors.

I like it. Raul654 12:20, 9 November 2007 (UTC)

Me too. Arthur 18:03, 9 November 2007 (UTC)

Thanks! henrik•talk 19:40, 9 November 2007 (UTC)

I would like the article to talk about dependencies, pipelining and vectorization in a slightly more general way (i.e. not tied to parallelism as in thread parallelism or a specific implementation in a processor), as well as using slightly different terminology. I've started to type up some notes at User:Henrik/sandbox/Parallel computing notes. But I'm a little bit hesitant to make so major modifications to a current WP:FAC article, so I thought I'd bring it here for discussion first. Your thoughts would be appreciated. henrik•talk 19:40, 9 November 2007 (UTC)

What do you think of this first line

Parallel computing is a form of computing in which multiple processors are used to allow many instructions to be carried out simultaneously.

I think it needs to be clear that multiple processors are required.

Also, is "compute resource" the correct term? I thought it was a typo but it is used repeatedly Mad031683 21:44, 14 November 2007 (UTC)

"processor" is (pretty much) synonymous with a von Neumann machine, but there are other methods of doing parallel computing, such as dataflow machines, ASIC or FPGA algorithm implementations, superscalar and vector processors (where there is parallelism within a single von Neumann control logic), et cetera. Granted, single von Neumanns are the vastly most prolific form of "compute resource", so if the consensus is that it is too pedantic to avoid the word "processor", I won't be the one starting an edit war :-) henrik•talk 21:54, 14 November 2007 (UTC)

The definition as quoted in the cited source says "processing elements" which is more accurate terminology, IMO. Raul654 23:18, 14 November 2007 (UTC)

Things left from the old FAC nom that need to be addressed before renominating:

1. Expand the history section. Links: http://ei.cs.vt.edu/~history/Parallel.html, http://ctbp.ucsd.edu/pc/html/intro4.html, http://www.gridbus.org/~raj/microkernel/chap1.pdf
2. Discuss more about the software side (languages)
3. Make references consistent

After those are done, renominate on FAC. Raul654 (talk) 20:36, 10 December 2007 (UTC)

This section is very odd. How is fine-grained and coarse-grained parallelism related to communication between subtasks at all? The granularity is strictly related to computation length, not communication frequency. —Preceding unsigned comment added by Joshphillips (talk • contribs) 22:38, 19 December 2007 (UTC)

False. Granularity is the amount of time/computation between communication events (or, more strictly speaking, the ratio of time between computation and communication) Raul654 (talk) 22:45, 19 December 2007 (UTC)

I have just finished reviewing this article for good article (GA) status. As it stands, I cannot pass it as a GA due to some issues outlined below. As such, I have put the nomination on hold, which allows up to seven days in which editors can address these problems before the nomination is failed without further notice. If and when the necessary changes are made, I will come back to re-review it.

GA review (see here for criteria)

1. It is reasonably well written.

   a (prose): b (MoS):

   The section on data parallelism needs some tidying. The main article on it explains it quite well, but the section in this article should summarise the main article. Additionally, the paragraph about loop carried dependencies needs further explanation and/or an example, as the text is currently unclear and the linked article is non-existent.
   
   The history section needs a bit of work also; I feel that it needs to be a bit more in-depth about the progression of parallel computing rather than just picking a couple of milestones to highlight.

2. It is factually accurate and verifiable.

   a (references): b (citations to reliable sources): c (OR):

   Seems fairly well-referenced to me. The only issue I have (which would not preclude me from passing it as a GA) is with the labelling of the various references to the two Hennessy/Patterson books: after the first full references (i.e. title, publisher etc), one is referred to as Hennessy and Patterson and the other as Patterson and Hennessy. Personally I find this a touch confusing but am unsure exactly how to solve it short of using full references in all of the citations (which is not ideal either).

3. It is broad in its coverage.

   a (major aspects): b (focused):

   Nicely covers the background and then focuses on the various topics well.

4. It follows the neutral point of view policy.

   Fair representation without bias:

   Article appears to be NPOV.

5. It is stable.

   No edit wars etc.:

   Recent history indicates the article to be stable.

6. It is illustrated by images, where possible and appropriate.

   a (images are tagged and non-free images have fair use rationales): b (appropriate use with suitable captions):

   There is no fair use rationale for Image:Cell Broadband Engine Processor.jpg (used in the section 'Multicore computing'). Additionally, I don't feel that the use of the image passes the fair use policy, most specifically section 8 (Non-free content is used only if its presence would significantly increase readers' understanding of the topic, and its omission would be detrimental to that understanding).

7. Overall:

   Pass/Fail:

   The article is certainly within reach of achieving GA status once the issues outlined above are addressed. My reasoning for putting it on hold, as opposed to failing it outright, is that I feel the sections I outlined above just need some expanding and tidying, which I feel could be achieved in the seven day hold period. I would welcome any comments or queries about the points I have raised, but would prefer them to be made here on the talk page (as opposed to my user talk) so that all editors involved with the article are aware of them. Blair - Speak to me 01:43, 6 January 2008 (UTC)

I have expanded the discussion of loop carried dependencies. Raul654 (talk) 19:00, 6 January 2008 (UTC)

One thing: the example for loop-carried dependencies seems wrong to me - it illustrates the point, but it doesn't seem to be calculating the Fibonacci numbers as stated. If I'm reading it correctly, PREV and CUR are initialised to 0 and 1 respectively. Inside the loop, PREV is then set to CUR i.e. PREV = 1. CUR is then calculated as PREV + CUR = 2. This then repeats to give PREV = 2, CUR = 4 and then PREV = 4, CUR = 8 and finally PREV = 8, CUR = 16 before the loop exits. In other words, the output (counting the initialisations) is 0, 1, 2, 4, 8, 16 as opposed to the expected Fibonacci series 0, 1, 1, 2, 3, 5, 8, 13.

The problem seems to be a read-after-write hazard. I would personally write the pseudocode for a Fibonacci series as

1: prev1 := 0
2: prev2 := 0
3: cur := 1
4: do:
5:    prev1 := prev2
6:    prev2 := cur
7:    cur := prev1 + prev2
8: while (cur < 10)

If my calculations are correct, this would then output the Fibonacci terms as expected. I would be bold and change this myself, but I would like to double-check that I am not misreading the current example. Additionally, I am not sure which is the better example - the existing one (if the text is corrected to match) is simpler to understand, however a good compiler would optimise it away. As an aside, it would probably end up as a left-shift as this corresponds to multiplying a binary-represented number by two, which is what the contents of the loop are doing.

Also, apologies for being a couple of days late with coming back and reviewing your changes - works been busy and I am currently hunting for a new flat to live in this year. It looks good to me; I am about to pass this as a good article. The example issue is relatively minor (I don't think many people will be investigating it in such detail as myself) and will be fixed shortly in whatever way is decided here (including the possible decision that I am blind and misreading it!). For my two cents, I would stick with the Fibonacci series idea.

Congratulations and thank you for your efforts on this article. Feel free to use one of the templates in Category:Wikipedia Good Article contributors on your userpage.

Blair - Speak to me 09:36, 16 January 2008 (UTC)

I don't think this redirect necessarily is appropriate: a distributed system is not just a computer concept (an organisation of employees is a distributed system). I think there should be a separate article, hyperlinking to this one, on the wider concept. Any thoughts? ElectricRay (talk) 10:51, 21 January 2008 (UTC)

Thank you for pointing this out. In fact, I would (and did) argue that a similar redirect is uncalled-for, even within the context of programming and computer science. See the Talk section above, under "Rewrite". I just checked what redirects to this article, and was surprised to discover that many other inappropriate redirects had previously escaped my notice. I believe these should also be removed, or moved to the appriate article(s). - JCLately (talk) 15:21, 21 January 2008 (UTC)

agree with the thrust of your posts above. I don't have the information to write a separate article (except by way of original research) but in terms of non-computing components the sort of thing I had in mind is this (and this is definitely WP:OR, so not suitable for the enyclopaedia itself):

in a non-digital context, a "network" might be used generically to denote any means by which any spatially dispersed members of single system or community interacts. No need for common purpose or intentionality (eg a city: it has no common purpose). The End-to-end principle is substrate-neutral and applies equally to any such system: An effective bottom layer of such a network needs (a) to preserve sufficient integrity and structure of movement/interpretation/communication so that there is as much flexibility to impose complexity on the edge of the network as possible and (b) the network is otherwise as agnostic as possible about those complexities.

Clearly one needs to define a minimum structure so that the system will work and thereafter (to optimise the ability to impose complexity), the highest common factor of all user needs of *all* feasible system users. (If you define your users and their needs in terms of, for example, negotiating derivatives transactions, you will come up with a bottom layer which looks like an ISDA Master Agreement, which gives huge flexibility within the context of a derivatives trading relationship but a fair bit of inbuilt (& functional) complexity too, which saves time at the periphery, as long as all you are doing is trading derivatives. If you tried to use ISDA as a system for broadcasting news, it would be far less functional.

Other examples of the "simple system; complex overlay":

• "Natural" languages (like English) and the speciality language that develops atop it (Legalese, computer jargon, local idioms and dialects)

• rail network against a road network - for users, the trains, carriages and time of delivery and destinations are fixed by the network, and individual users don't have much choice/flexibility about how they are used. This works well where there are large number of users all of whom need to move between a limited number of discrete points in predictable numbers at predictable times. But a train service wouldn't work so well in a relatively low density population with no central hub point (or an insufficently limited number of points of usual travel) (perhaps one reason why there's no metropolitan rail service in Los Angeles).

• By contrast a road network is cheaper to maintain and run (less complexity in base network: no prescribed means of conveyance on the network and relatively little constraint on usage (traffic lights etc); scope for greater flexibility/complexity on the part of the user), but is in a way more prescriptive on the user in terms of rules of the road (these are mostly transparent to. Note a user can create a greater level of system complexity (akin to a rail service) on top of the road network - by running a bus service - which might not be quite as efficient as a rail service (in the right population environment) but is cheaper and doesn't (much) interfere with the residual flexibility of users to design other means of using the nework (in that they are also free to walk, run, ride, cycle, or drive a hovercraft as long as they comply with minimum rules of the road).

• Challenge is to craft suitable rules of the road that prevent cars and buses colliding but otherwise enable free and fast movement of traffic.

• Owners of various parts of the network can overlay their own rules if they wish as long as a user who doesn't need them has another way of getting to her destination (the beauty of a nodal network)

I'd be astounded if I were the first person to ever make that connection (Lawrence Lessig has done something similar in Code Version 2.0), but I don't have the background to know who it was. I sure would like to, though. ElectricRay (talk) 09:57, 22 January 2008 (UTC)

I'm tempted to delete a few occurrances of this word and just write "computers" as in:

• Some parallel computers architectures use smaller.....
• While computers architectures to deal with this were devised... GrahamColm^Talk 13:20, 4 May 2008 (UTC)

1. Program runtime can not decrease in linear with increasing the number of processors even in ideal case (as it is shown in "Parallelization diagram").
2. Does the article make any distinction between words "serial" and "sequential" in the context of computing paradigm?
3. Do you consider important to mention any theoretical model for parallel computing (PRAM for example)?
4. Article lacks at least to mention some impossibility results in the context of P-Completness theory and the notion of inherently sequential problems, do you agree?, kuszi (talk) 07:42, 17 May 2008 (UTC).

To answer your questions: (1) You're wrong. In the ideal case, a program has no sequential bottleneck and parallelizes with 100% efficiency. (2) There is no difference between "sequential" and "serial" processing. The article uses both terms interchangeably. (3) No, mathematical models of parallel computing are a dime a dozen. (4) I don't understand this question. Raul654 (talk) 17:56, 17 May 2008 (UTC)

(1) Are you sure? Even when efficiency is 100% it is difficult to have runtime below 0. (2)Thank you. (3,4) Possibly the mathematical models for parallel computing are easy to find, however it would be nice to at least mention to the reader that such models exists and that we can, for example, distinguish between NC and P-complete problems. 85.128.91.247 (talk) 05:15, 19 May 2008 (UTC). I was logged out, sorry, kuszi (talk) 10:21, 19 May 2008 (UTC).

One more thing:

1. Last sentence in the introduction: The speedup of a program as a result of parallelization is given by Amdahl's law. - objection here, possibly the potential speedup or something alike? kuszi (talk) 10:21, 19 May 2008 (UTC).

This article uses the terms speed-up and speedup. I think they're both ugly, but whichever is to be preferred, then the article ought to be consistent. --Malleus Fatuorum (talk) 03:36, 15 June 2008 (UTC)

Seeing as this is a featured article, it should be as perfect as possible:

• I think it would be beneficial to expand the lead so that it fully summarizes the article, to comply with WP:LEAD.
  • The lead is appropriate for an article of this length, per Wikipedia:Lead ("> 30,000 characters ... three or four paragraphs"). It hits on all the important points without getting bogged down in details that are better handeled later in the article. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • The lead neglects several facts from the 'History' section. A general rule of thumb is that every section should be summarized in the lead. I also don't see any mention of Amdahl's law and Gustafson's law, and Moore's Law. Instead, the lead focuses on the different types of parallelism, the hardware/software and uses too many technical terms, even though WP:LEAD states "In general, specialized terminology should be avoided in an introduction.". — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
      • Amdahl's law is mentioned in the last sentence of the lead; Gustafon's doesn't need to be - it makes a finer, more subtle point than Amdahl's law. Moore's law doesn't need to be mentioned because the lead already mentions the trend towards parallelism (It has been used for many years, mainly in high-performance computing, but interest in it has grown in recent years due to the physical constraints preventing frequency scaling.). Moore's law is needed to explain the reason for this, but that's too much detail for the lead. The lead focuses on hardware and software aspects because that's what the article does. It also happens to be a naturally a good way to divide up the the topic. As for technical terms, the lead itself can be understood by someone who doesn't understand most of the technical terms. One needn't know what a cluster is to get the gist of the sentence that mentions them. A number of people on the FAC commentary page noted that they thought it did good job of explaining the subject to non-experts. Raul654 (talk) 20:21, 16 July 2008 (UTC)

• Completely unreferenced paragraph: "Moore's Law is the empirical observation that transistor density in a microprocessor doubles every 18 to 24 months. Despite power consumption issues, and repeated predictions of its end, Moore's law is still in effect. With the end of frequency scaling, these additional transistors (which are no longer used for frequency scaling) can be used to add extra hardware for parallel computing."
  • I've added a reference to Moore's original 1965 presentation. Raul654 (talk) 19:42, 16 July 2008 (UTC)
• Another: "Theoretically, the speed-up from parallelization should be linear—doubling the number of processing elements should halve the runtime, and doubling it a second time should again halve the runtime. However, very few parallel algorithms achieve optimal speed-up. Most of them have a near-linear speed-up for small numbers of processing elements, which flattens out into a constant value for large numbers of processing elements."
  • The fact that doubling the processing power should halve the runtime is common knowledge and does not require a citation. The fact that that few applications achieve this optimality for a large number of threads is also common knowledge. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • It says "Theoretically", so it deserves a footnote. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
      • You could replace the word "Theoretically" with "Under ideal conditions" - it'd still mean the same thing. That does not mean it requires a footnote. Raul654 (talk) 20:21, 16 July 2008 (UTC)
• No reference for "Amdahl's law assumes a fixed-problem size and that the size of the sequential section is independent of the number of processors, whereas Gustafson's law does not make these assumptions."
  • This follows from simple definitions already given and cited in the article. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • I must have missed something. Where is the reference for this? — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
      • Reference 10 defines Amdahl's law, and reference 12 defines Gustafson's law. Raul654 (talk) 20:21, 16 July 2008 (UTC)
• And "Understanding data dependencies is fundamental in implementing parallel algorithms. No program can run more quickly than the longest chain of dependent calculations (known as the critical path), since calculations that depend upon prior calculations in the chain must be executed in order. However, most algorithms do not consist of just a long chain of dependent calculations; there are usually opportunities to execute independent calculations in parallel."
  • The first two sentences come from the definition of critical path. For the last sentence, I'm sure there are plenty of references that could be put there (studies analyzing the possible parallelization of one or more classes of application) but I can't think of any specifics off the top of my head. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • I think it would be best if <ref name=""> was used more in this article to ensure its fully referenced. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
• I could continue giving examples, but there are too many. It's a general rule of thumb that every statistic, claim, fact, quote, etc should be given its own footnote to satisfy FA criteria 1c.
  • Quotes and statistics, yes. Claims - no. General and subject specific common knowledge need not be cited, per Wikipedia:When to cite Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • Do you really think "specific common knowledge" applies when it comes to parallel computing? Most people don't have a clue of such a technical topic, and we should always try to make things as easy to read and understand as possible. Anyone can make a claim to general knowledge—but that doesn't mean it's accurate or true at all. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
• 'Applications' section is a list, when it could be prose to satisfy FA criteria 1a.
  • We don't put in prose just for the sake of putting in prose. Prose is generally preferable to lists because a list doesn't allow the author to compare and contrasts the items. On the other hand, the list in this article was put there because they don't have anything in common, insofar as the algorithms to solve them are concerned. There's nothing to say about each item besides a description of what it is, and how it gets solved, and that's appropriately done in their respective articles. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • A fair point, though prose would still fit in better. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
• The History section covers very little detail, and suggests readers look at History of computing, an awful stubby article with no detail on parallel computing at all. What good is that, exactly? - Means the article possibly fails FA criteria 1b for comprehensiveness.
  • The History of computing article is sub-par because its authors unwisely decided to fork it off into *nineteen* seperate sub-articles instead of writing one good one. This article gives a good overview of the history of parallel computing hardware; beyond that, however, is beyond that scope of this article. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • I would remove the link to History of computing, which does not even mention parallel computing. Instead, there should be a History of parallel computing article for readers to look at. Still, I am disappointed with the History section and since there are literally thousands of reliable sources to fill it out a bit, it certainly wouldn't hurt, and it would help this fully meet the criteria for comprehensiveness. At the moment, the History section looks like an afterthought stuck on at the end. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
• I have noticed the author names in the citations are incorrectly formatted. It should be last, first, not first, last.
  • No, it need not be: All citation techniques require detailed full citations to be provided for each source used. Full citations must contain enough information for other editors to identify the specific published work you used. There are a number of styles used in different fields. They all include the same information but vary in punctuation and the order of the author's name, publication date, title, and page numbers. Any of these styles is acceptable on Wikipedia so long as articles are internally consistent. - http://en.wikipedia.org/wiki/Wikipedia:Citing_sources#Citation_styles Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • That is correct, but 99.9% of FAs I have seen use the latter style—give readers what they're used to (and what is commonly used by academics). — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
      • The reference formatting was previously consistent, but I redid them to the more common format anyway. SandyGeorgia (Talk) 20:14, 16 July 2008 (UTC)
• What makes http://www.webopedia.com/TERM/c/clustering.html a reliable source?
  • It's affiliated with internet.com (a respectable website in itself) and Jupitermedia Raul654 (talk) 19:42, 16 July 2008 (UTC)
• Why are page numbers given after the ISBN instead of before it? Again, another example that the citations are not formatted as they should be.
  • The citations are given in precedence order - the information required to find the work, then the page itself. This is acceptable per the above statement, "Any of these styles is acceptable on Wikipedia so long as articles are internally consistent" Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • Correct again, but this is the first FA I've come across which uses this style. (I frequently review articles at FAC). — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)
• In all, this article has 43 refs—it's quite a long article, and the number of refs does not correspond with the amount of prose. — Wackymacs (talk ~ edits) 07:55, 13 July 2008 (UTC)
  • There is no requirement that we have to have X dozen references in a featured article. It's not a contest to jam in the most possible references. Raul654 (talk) 19:42, 16 July 2008 (UTC)
    • My rule of thumb is that more references is better than fewer, and this article does have fewer than expected, and many unreferenced paragraphs which I did not expect. This especially necessary when it comes to articles covering technology subjects which readers often find difficult to understand. — Wackymacs (talk ~ edits) 20:05, 16 July 2008 (UTC)

Article says: "Slotnick had proposed building a massively parallel computer for the Lawrence Livermore National Laboratory" - this is quite misleading, as ILLIAC IV has been build at the University of Illinois. kuszi (talk) 09:07, 13 August 2008 (UTC).

Its a poor example, i.e. late and slow, While Richard Feynman was implementing the concepts, with human calculators and punched cards is Los Almos.

Article says: "With the end of frequency scaling, these additional transistors (which are no longer used for frequency scaling) can be used to add extra hardware for parallel computing." - it suggests that additional transistors where (before 2004) needed for frequency scaling - rather nonsense. kuszi (talk) 09:24, 13 August 2008 (UTC).

Article says: "In 1967, Amdahl and Slotnick published a debate about the feasibility of parallel processing at American Federation of Information Processing Societies Conference" - probably you mean Amdahl's article: "Gene Amdahl,Validity of the single processor approach to achieving large-scale computing capabilities. In: Proceedings of the American Federation Information Processing Society, vol. 30, 1967, 483–485."? kuszi (talk) 09:24, 13 August 2008 (UTC).

Michael R. D'Amour was never CEO of DRC. He was not around when we worked with AMD on the socket stealer proposal. I came up with the idea of going into the socket in 1996 (US Patent 6178494, Issued Jan 2001). 16 days after the Opteron was announced I had figured out how to replace the Opteron with an FPGA. I personally did the work to architect the board, fix the Xilinx non-working Hypertransport verilog code and get the (then) linux BIOS to boot the FPGA in the Opteron socket. I was also responsible for figuring out if current FPGA's could work in the high performance last generation of the FSB. This lead to the Quick path program. We opted out of the FSB program as it is a dead bus. Beercandyman (talk) 20:57, 16 February 2009 (UTC) Steve Casselman CTO and Founder DRC Computer.

OK, from this and this it looks like he's the COO, not CEO. Raul654 (talk) 00:46, 19 February 2009 (UTC)

As power consumption by computers has become a concern in recent years,[3] parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multicore processors.[4]

The sentence is missing a clause (and source) to relate "power consumption is a concern" to "parallel computing is dominant". It goes on to say that the dominance applies to "computer" architecture, "mainly" in processors. Are we sure that it isn't "only" in processors? It seems that nearly every other component has been moving towards serial architectures, including USB, Firewire, Serial ATA, and PCI Express, at the expense of the previous parallel standard. The serial communications and parallel communications articles have more information, and a very in-depth comparison can be found in this HyperTransport white paper: [1] (HT combines serial and parallel). Ham Pastrami (talk) 06:49, 18 March 2009 (UTC)

(A) The sentence is grammatically correct. There is no missing clause there. (B) Power consumption concerns (under the general aegis of "power aware computing") apply to the entire computer system. (C) Parallelism impacts the whole of the system, including the need for better buses and distributed storage systems. The very "serial" bus you site, Hyper-transport, was designed from the ground up to enable system parallelism (multiple AMD processors or, in DRC's case, an AMD processor and an FGPA accelerator). In short, the fact that a few buses have (for the time being, at the bit level) switched from parallel to serial is a triviality. (D) Please name a single other component that is going towards serialization. Buses are being redesigned to enable parallelism (both in protocols, support for cache coherency, the development of optical and opto-electronic interconnects, etc), memory is being redesigned to enable parallelism (e.g, Processor-in-memory technologies). Raul654 (talk) 07:08, 18 March 2009 (UTC)

(A)Grammar is not the issue, (B)the lack of information is. What does power have to do with parallelism? (C)Please read again, I did not cite HT as a "serial" bus. If, as you claim, parallelism (of the processor, ostensibly) impacts the computer as a whole, why is this not the claim in the article, which explicitly states that it "mainly" affects multicore processors? You'll pardon me if I don't agree that the mass migration of buses to serial interfaces is a "triviality", though I will accept henrik's explanation. (D)I thought I provided several examples above (though you appear to distinguish between "component" and "bus"). The article does not say "component", it says "computer architecture", of which buses are still quite an important part. Any way you want to look at it, there is a mismatch in the language. Either you mean parallelism is dominant in something else, or you don't mean that parallelism is dominant in computer architecture. Ham Pastrami (talk) 02:49, 19 March 2009 (UTC)

What does power have to do with parallelism? - this is discussed at length in the article (the first 5 paragraphs in the background section). In short:

P=C*V^2*F,

and

FLOPS ~ F * #cores

From these equations, it is obvious that the best way to increase performance while maintaining or minimizing heat is to keep frequency constant (or lower it) while increasing the number of cores.

why is this not the claim in the article, which explicitly states that it "mainly" affects multicore processors? - it *doesn't* say it affects mainly multicore processors. Parallelism affects most if not all aspects of the system design (especially, but not limited to, the processors, memory, and the interconnects in between). What the article says is that dominant paradigm in computer architecture is the move to multicore. And the article discusses the impact of parallelism on software, processors, memory, and network topology (which includes buses).

I thought I provided several examples above (though you appear to distinguish between "component" and "bus"). The article does not say "component", it says "computer architecture", of which buses are still quite an important part. Any way you want to look at it, there is a mismatch in the language. - I have no idea what this means.

Either you mean parallelism is dominant in something else, or you don't mean that parallelism is dominant in computer architecture. - I have no idea where this claim comes from, but you are utterly and completely wrong. Parallelism is most certainly the dominant paradigm in computer architecture. Raul654 (talk) 03:07, 19 March 2009 (UTC)

Serial buses are a completely different beast than parallel computation, it makes very little sense to compare the two. The serialization of buses is mostly due to development of SerDes multi-gigabit transceivers and physical constraints of long parallel buses. The parallelization of processors (and thusly computing in common language) is also due to physical constraints, but entirely different ones. henrik•talk 08:11, 18 March 2009 (UTC)

I can understand that, but the article's claim is "dominance" in "computer architecture". Though buses and parallel processors may be different beasts, they are all part of computer architecture. So the issue remains that the statement is too bold for the given rationale. Ham Pastrami (talk) 02:49, 19 March 2009 (UTC)

What does the "embarassingly" mean in: "grid computing typically deals only with embarrassingly parallel problems". Does it just mean "highly"? There must be a better word for it. Embarassingly sounds POV, as it makes me wonder who is embarassed - presumably the people trying to cure cancer don't think it is an embarassing problem? Maybe Seti@home users should be embarassed, but i doubt they are.

I didn't change it as i havn't read the source, and i'm not certian of what it should mean.YobMod 08:43, 18 March 2009 (UTC)

Please see Parallel computing#Fine-grained, coarse-grained, and embarrassing parallelism and Embarrassingly parallel. "embarrassingly parallel" is a standard term in the parallel-computing community. But perhaps the phrase should also be wikilinked in the grid computing section if it's causing confusion. --Allan McInnes (talk) 09:06, 18 March 2009 (UTC)

That's quite irresponsible to be on the front page; it's not power consumption the main reason of success of multi-core processors but simply the inability to produce faster processors in the pace (comparable to the 80s and 90s) needed to sell to the public "shiny new super faster computers compared your last one that you must buy now or you're behind the times". Shortening the transistor size is slower and parallel computing offers another route to "shiny new faster personal computers". The combination of smaller transistors and multicore processors is the new deal. --AaThinker (talk) 10:00, 18 March 2009 (UTC)

The inability to produce faster processors (by increasing clock frequency) was to a large extent due to power consumption. henrik•talk 10:27, 18 March 2009 (UTC)

no, it was due to electronic leaks. --AaThinker (talk) 10:36, 18 March 2009 (UTC)

..which caused unacceptable power consumption and cooling problems. This is not primarily an electronics oriented article, "power consumption" has been judged to be an acceptable compromise between accuracy and overwhelming the readers with details. henrik•talk 10:41, 18 March 2009 (UTC)

but the point is, energy aside, they could not make computers as we're used to if they had to make a cooler as big as a fridge. --AaThinker (talk) 12:31, 18 March 2009 (UTC)

Parasitic capacitance and inductance (which are inversely related to size -- smaller, more closely transistors suffer from these effects worse) were one cause of the heat problem, but the linear relationship between frequency and power consumption also played a substantial, if not dominant, role in ending frequency scaling as the major driving force in processor speedup. Raul654 (talk) 20:00, 18 March 2009 (UTC)

Is the Cray-2 image the right one for this article? It is a vector processor, while a MPP like the Intel Paragon, CM-2, ASCI Red or Blue Gene/P might be better illustrations of parallel computing. -- Autopilot (talk) 12:43, 18 March 2009 (UTC)

Modern processor instruction sets do include some vector processing instructions, such as with AltiVec and Streaming SIMD Extensions (SSE).

This is conflating vector math with vector processing, isn't it? Can someone knowledgeable remove the above line if it's not talking about Cray-1 style vector computers? Tempshill (talk) 16:20, 18 March 2009 (UTC)

Vector math can be implemented using a vectorized ISA (like happens in a GPU), so they are related, but not the same thing. This is conflating vector math with vector processing, isn't it? - no, it's not. See the example listed in the SSE article. Raul654 (talk) 19:21, 18 March 2009 (UTC)

Surprised there was no mention of Thinking Machines' Connection Machine, not even in the history section. --IanOsgood (talk) 22:19, 18 March 2009 (UTC)

 Yes, the first thing that comes to mind when you mention 
parallel processing is the Hillis Connection Machine. It 
failed in business only because it was ahead of it's time.
 Should get a mention. blucat - David Ritter  —Preceding unsigned comment added by 198.142.44.68 (talk) 15:24, 21 March 2009 (UTC) 

Category:Parallel computing is itself a category within Category:Concurrent computing — Robert Greer (talk) 23:15, 18 March 2009 (UTC)

The most common type of cluster is the Beowulf cluster

I think there should be a reference cited for this, otherwise this is just an opinion of an editor. --66.69.248.6 (talk) 17:54, 21 March 2009 (UTC)

Please see Talk:Distributed computing/Archive 1#What is parallel computing? JonH (talk) 22:53, 10 August 2009 (UTC)

Does anyone object to me setting up automatic archiving for this page using MizaBot? Unless otherwise agreed, I would set it to archive threads that have been inactive for 60 days.--Oneiros (talk) 13:13, 21 December 2009 (UTC)

  I am green as a freshly minted Franklin, never posted before (so be nice)

  Graduate student at the University of Illinois at Urbana-Champaign
  Semester project; regardless, always wanted to do something like this...
  All initial work should be (majority) my effort
  As a word to the wise is sufficient; please advise, rather than take first-hand action.

  The article should (and will) be of substantial size; but is currently no more that scaffolding
  The "bullet-points" are intended to outline the potential discussion, and will NOT be in the finished product
  The snippet of text under each reference is from the reference itself, to display applicability
  Direct copying of reference information WILL NOT be part of any section of this article
  Again, this information is here to give an idea of the paper, without having to go and read it...

  Article sections that are drafted so far are quite "wordy".... (yawn...)
  Most of the prose at this point has about a 1.5X - 2.0X inflated over the anticipated final product
  This is my writing style, which has a natural evolution, through iteration
  Complex -> confused -> constrained -> coherent -> concise (now, if it only took 5 iterations???)

  Again, thank you in advance for you patience and understanding
  I look forward to working with you guys...
  Project Location: Distributed operating system
  Project Discussion: Talk: Distributed operating system

JLSjr (talk) 01:31, 20 March 2010 (UTC)

I've uploaded an audio recording of this article for the Spoken Wikipedia project. Please let me know if I've made any mistakes. Thanks. --Mangst (talk) 20:20, 1 November 2010 (UTC)

Should the paragraph about Application Checkpointing be in this article about parallel computing?

I think it's not a core part of parallel computing but a part of the way applications work and store their state. Jan Hoeve (talk) 19:33, 8 March 2010 (UTC)

Fault tolerance is a major (though often overlooked) part of parallel computing, and checkpointing is a major part of fault tolerance. So yes, it definitely belongs here. Raul654 (talk) 20:07, 8 March 2010 (UTC)

I came to the page to read the article and was also confused as to why checkpointing was there. It seems very out of place, and while fault tolerance may be important to parallelism, this isn't an article about fault tolerance mechanisms. It would be more logical to mention that parallelism has a strong need for fault tolerance and then link to other pages on the topic. 66.134.120.148 (talk) 01:27, 23 April 2011 (UTC)

What a pleasant surprise. A Wikipedia article on advanced computing that is actually in good shape. The article structure is (surprise) logical, and I see no major errors in it. But the sub-articles it points to are often low quality, e.g. Automatic parallelization, Application checkpointing, etc.

The hardware aspects are handled better here than the software issues, however. The Algorithmic methods section can do with a serious rework.

Yet a few logical errors still remain even in the hardware aspects, e.g. computer clusters are viewed as not massively parallel, a case invalidated by the K computer, of course.

The template used here called programming paradigms, is however, in hopeless shape and I will remove that given that it is a sad spot on an otherwise nice article. History2007 (talk) 22:40, 8 February 2012 (UTC)

Can we agree that parallel computing isn't the same as concurrent computing? See https://en.wikipedia.org/wiki/Concurrent_computing#Introduction — Preceding unsigned comment added by Mister Mormon (talk • contribs) 16:47, 3 February 2016 (UTC)

"The origins of true (MIMD) parallelism go back to Federico Luigi, Conte Menabrea and his "Sketch of the Analytic Engine Invented by Charles Babbage".[45][46][47]"

Not that I can see. This single mention refers to a system that does not appear in any other work, did not appear in Babbage's designs, and appears to be nothing more than "it would be nice if..." Well of course it would be. Unless someone has a much better reference, one that suggests how this was to work, I remain highly skeptical that the passage is correct in any way. Babbage's design did have parallelism in the ALU (which is all it was) but that is not parallel computing in the modern sense of the term. Maury Markowitz (talk) 14:25, 25 February 2015 (UTC)

Dear Maury Markowitz,

Forgive me for reverting a recent edit you made to the parallel computing article.

You are right that Babbage's machine had a parallel ALU, but does not have parallel instructions or operands and so does not meet the modern definition of the term "parallel computing".

However, at least one source says "The earliest reference to parallelism in computer design is thought to be in General L. F. Menabrea's publication ... It does not appear that this ability to perform parallel operation was included in the final design of Babbage's calculating engine" -- Hockney and Jesshope, p. 8. (Are they referring to the phrase "give several results at the same time" in (Augusta's translation of) Menabrea's article?)

So my understanding is that source says that the modern idea of parallel computing does go back at least to Menabrea's article, even though the idea of parallel computing was only a brief tangent in Menabrea's article whose main topic was a machine that does not meet the modern definition of parallel computing.

Perhaps that source is wrong. Can we find any sources that disagree? The first paragraph of the WP:VERIFY policy seems to encourage presenting what the various sources say, even when it is obvious that some of them are wrong. (Like many other aspects of Wikipedia, that aspect of "WP:VERIFY" strikes me as crazy at first, but then months later I start to think it's a good idea).

The main problem I have with that sentence is that implies that only MIMD qualifies as "true parallelism". So if systolic arrays (MISD) and the machines from MasPar and Thinking Machines Corporation (SIMD) don't qualify as true parallel computing, but they are not sequential computing (SISD) either, then what are they? Is the "MIMD" part of the sentence supported by any sources? --DavidCary (talk) 07:04, 26 February 2015 (UTC)

The "idea" may indeed date back to Menabrea's article, in the same way that flying to the Moon dates to Lucian's 79BC story about a sun-moon war. I think we do the reader a major disservice if we suggest that Menabrea musings were any more serious than Lucian's. Typically I handle these sorts of claims in this fashion...

"Menabrea's article on Babbage's Analytical Engine contains a passage musing about the potential performance improvements that might be achieved if the machine was able to perform calculations on several numbers at the same time. This appears to be the first historical mention of the concept of computing parallelism, although Menabrea does not explain how it might be achieved, and Babbage's designs did not include any sort of functionality along these lines."

That statement is factually true and clearly explains the nature of the post. Frankly, I think this sort of trivia is precisely the sort of thing we should expunge from the Wiki (otherwise we'd have mentions of Tesla in every article) but if you think it's worthwhile to add, lets do so in a form that makes it clear. Maury Markowitz (talk) 14:44, 26 February 2015 (UTC)

Seems to me that parallel means different things when discussing software and hardware, yet we somehow lump both into Computer Science. Since software parallelization is currently getting more interest, the term is easier to attach to that. But in a discussion on computing hardware, there are bit serial, or digit (in some radix) serial hardware. Also, in I/O buses, there is bit serial (like ethernet and SATA), and parallel like PCI and IDE/ATA/PATA. (Some schools now have CSE, Computer Science and Engineering, departments, which makes it more obvious that two different things are included.) Gah4 (talk) 17:08, 8 September 2016 (UTC)

I'm concerned that asynchronous programming redirects to this page. Asynchronous programming/computing is not the same as parallel computing. For example, JavaScript engines are asynchronous, but single-threaded (less web workers), meaning that tasks do not actually run in parallel, though they are still asynchronous. To my knowledge, that is how it works, and that is how NodeJS works, browsers, and Nginx. They are all single-threaded, yet asynchronous, and so not parallel. — Preceding unsigned comment added by 2605:A601:64C:9B01:7083:DDD:19AF:B6B7 (talk) 03:40, 14 February 2016 (UTC)

OK, but properly written asynchronous single thread code should still run in parallel if the system has that ability. There is the problem of properly debugging asynchronous code on single thread systems. Some bugs might not be found, or take longer to find. Gah4 (talk) 17:13, 8 September 2016 (UTC)

There is an article about Asynchrony (computer programming), but asynchronous programming redirects to parallel computing instead. Jarble (talk) 18:38, 5 September 2016 (UTC)

It does seem strange, but Asynchrony (computer programming) is pretty short, and doesn't say all that much. I was actually looking for an article describing asynchronous I/O, which is a little different than general asynchronous programming. There is enough overlap between parallel computing and asynchronous computing that it probably isn't so bad to redirect here. Gah4 (talk) 16:58, 8 September 2016 (UTC)

Hello fellow Wikipedians,

I have just modified 6 external links on Parallel computing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20081209154000/http://www.upcrc.illinois.edu/documents/UPCRC_Whitepaper.pdf to http://www.upcrc.illinois.edu/documents/UPCRC_Whitepaper.pdf
• Added archive https://web.archive.org/web/20080414141000/http://users.ece.utexas.edu/~patt/Videos/talk_videos/cmu_04-29-04.wmv to http://users.ece.utexas.edu/~patt/Videos/talk_videos/cmu_04-29-04.wmv
• Added archive https://web.archive.org/web/20071114212716/http://www.top500.org/stats/list/29/archtype to http://www.top500.org/stats/list/29/archtype
• Added archive https://web.archive.org/web/20080131221732/http://www.future-fab.com/documents.asp?grID=353&d_ID=2596 to http://www.future-fab.com/documents.asp?grID=353&d_ID=2596
• Added archive https://web.archive.org/web/20021012122919/http://wotug.ukc.ac.uk/parallel/ to http://wotug.ukc.ac.uk/parallel/
• Added archive https://web.archive.org/web/20100122110043/http://ppppcourse.ning.com/ to http://ppppcourse.ning.com/

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 14:59, 20 May 2017 (UTC)

Hello fellow Wikipedians,

I have just modified 4 external links on Parallel computing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20080218224945/http://download.intel.com/museum/Moores_Law/Articles-Press_Releases/Gordon_Moore_1965_Article.pdf to ftp://download.intel.com/museum/Moores_Law/Articles-Press_Releases/Gordon_Moore_1965_Article.pdf
• Added archive https://web.archive.org/web/20070927040654/http://www.scl.ameslab.gov/Publications/Gus/AmdahlsLaw/Amdahls.html to http://www.scl.ameslab.gov/Publications/Gus/AmdahlsLaw/Amdahls.html
• Added archive https://web.archive.org/web/20080131205427/http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=65878 to http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=65878
• Added archive https://web.archive.org/web/20081020052247/http://www.upcrc.illinois.edu/ to http://www.upcrc.illinois.edu/

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 17:20, 24 September 2017 (UTC)

Hello fellow Wikipedians,

I have just modified one external link on Parallel computing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20080705232106/http://view.eecs.berkeley.edu/ to http://view.eecs.berkeley.edu/

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 09:34, 7 October 2017 (UTC)