IBM Blue Gene

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/IBM_Blue_Gene 

IBM Blue Gene

A Blue Gene/P supercomputer at Argonne National Laboratory
Developer	IBM
Type	Supercomputer platform
Release date	BG/L: Feb 1999BG/P: June 2007BG/Q: Nov 2011
Discontinued	2015
CPU	BG/L: PowerPC 440BG/P: PowerPC 450BG/Q: PowerPC A2
Predecessor	IBM RS/6000 SP;QCDOC
Successor	IBM PERCS

Hierarchy of Blue Gene processing units

Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the petaFLOPS (PFLOPS) range, with low power consumption.

The project created three generations of supercomputers, Blue Gene/L, Blue Gene/P, and Blue Gene/Q. During their deployment, Blue Gene systems often led the TOP500 and Green500 rankings of the most powerful and most power efficient supercomputers, respectively. Blue Gene systems have also consistently scored top positions in the Graph500 list. The project was awarded the 2009 National Medal of Technology and Innovation.

As of 2015, IBM seems to have ended the development of the Blue Gene family though no public announcement has been made. IBM's continuing efforts of the supercomputer scene seems to be concentrated around OpenPower, using accelerators such as FPGAs and GPUs to battle the end of Moore's law.

History

In December 1999, IBM announced a US$100 million research initiative for a five-year effort to build a massively parallel computer, to be applied to the study of biomolecular phenomena such as protein folding. The project had two main goals: to advance our understanding of the mechanisms behind protein folding via large-scale simulation, and to explore novel ideas in massively parallel machine architecture and software. Major areas of investigation included: how to use this novel platform to effectively meet its scientific goals, how to make such massively parallel machines more usable, and how to achieve performance targets at a reasonable cost, through novel machine architectures. The initial design for Blue Gene was based on an early version of the Cyclops64 architecture, designed by Monty Denneau. The initial research and development work was pursued at IBM T.J. Watson Research Center and led by William R. Pulleyblank.

At IBM, Alan Gara started working on an extension of the QCDOC architecture into a more general-purpose supercomputer: The 4D nearest-neighbor interconnection network was replaced by a network supporting routing of messages from any node to any other; and a parallel I/O subsystem was added. DOE started funding the development of this system and it became known as Blue Gene/L (L for Light); development of the original Blue Gene system continued under the name Blue Gene/C (C for Cyclops) and, later, Cyclops64.

In November 2004 a 16-rack system, with each rack holding 1,024 compute nodes, achieved first place in the TOP500 list, with a Linpack performance of 70.72 TFLOPS. It thereby overtook NEC's Earth Simulator, which had held the title of the fastest computer in the world since 2002. From 2004 through 2007 the Blue Gene/L installation at LLNL gradually expanded to 104 racks, achieving 478 TFLOPS Linpack and 596 TFLOPS peak. The LLNL BlueGene/L installation held the first position in the TOP500 list for 3.5 years, until in June 2008 it was overtaken by IBM's Cell-based Roadrunner system at Los Alamos National Laboratory, which was the first system to surpass the 1 PetaFLOPS mark. The system was built in Rochester, MN IBM plant.

While the LLNL installation was the largest Blue Gene/L installation, many smaller installations followed. In November 2006, there were 27 computers on the TOP500 list using the Blue Gene/L architecture. All these computers were listed as having an architecture of eServer Blue Gene Solution. For example, three racks of Blue Gene/L were housed at the San Diego Supercomputer Center.

While the TOP500 measures performance on a single benchmark application, Linpack, Blue Gene/L also set records for performance on a wider set of applications. Blue Gene/L was the first supercomputer ever to run over 100 TFLOPS sustained on a real-world application, namely a three-dimensional molecular dynamics code (ddcMD), simulating solidification (nucleation and growth processes) of molten metal under high pressure and temperature conditions. This achievement won the 2005 Gordon Bell Prize.

In June 2006, NNSA and IBM announced that Blue Gene/L achieved 207.3 TFLOPS on a quantum chemical application (Qbox). At Supercomputing 2006, Blue Gene/L was awarded the winning prize in all HPC Challenge Classes of awards. In 2007, a team from the IBM Almaden Research Center and the University of Nevada ran an artificial neural network almost half as complex as the brain of a mouse for the equivalent of a second (the network was run at 1/10 of normal speed for 10 seconds).

Name

The name Blue Gene comes from what it was originally designed to do, help biologists understand the processes of protein folding and gene development. "Blue" is a traditional moniker that IBM uses for many of its products and the company itself. The original Blue Gene design was renamed "Blue Gene/C" and eventually Cyclops64. The "L" in Blue Gene/L comes from "Light" as that design's original name was "Blue Light". The "P" version was designed to be a petascale design. "Q" is just the letter after "P". There is no Blue Gene/R.

Major features

The Blue Gene/L supercomputer was unique in the following aspects:

• Trading the speed of processors for lower power consumption. Blue Gene/L used low frequency and low power embedded PowerPC cores with floating point accelerators. While the performance of each chip was relatively low, the system could achieve better power efficiency for applications that could use large numbers of nodes.
• Dual processors per node with two working modes: co-processor mode where one processor handles computation and the other handles communication; and virtual-node mode, where both processors are available to run user code, but the processors share both the computation and the communication load.
• System-on-a-chip design. Components were embedded on a single chip for each node, with the exception of 512 MB external DRAM.
• A large number of nodes (scalable in increments of 1024 up to at least 65,536)
• Three-dimensional torus interconnect with auxiliary networks for global communications (broadcast and reductions), I/O, and management
• Lightweight OS per node for minimum system overhead (system noise).

Architecture

The Blue Gene/L architecture was an evolution of the QCDSP and QCDOC architectures. Each Blue Gene/L Compute or I/O node was a single ASIC with associated DRAM memory chips. The ASIC integrated two 700 MHz PowerPC 440 embedded processors, each with a double-pipeline-double-precision Floating Point Unit (FPU), a cache sub-system with built-in DRAM controller and the logic to support multiple communication sub-systems. The dual FPUs gave each Blue Gene/L node a theoretical peak performance of 5.6 GFLOPS (gigaFLOPS). The two CPUs were not cache coherent with one another.

Compute nodes were packaged two per compute card, with 16 compute cards plus up to 2 I/O nodes per node board. There were 32 node boards per cabinet/rack. By the integration of all essential sub-systems on a single chip, and the use of low-power logic, each Compute or I/O node dissipated low power (about 17 watts, including DRAMs). This allowed aggressive packaging of up to 1024 compute nodes, plus additional I/O nodes, in a standard 19-inch rack, within reasonable limits of electrical power supply and air cooling. The performance metrics, in terms of FLOPS per watt, FLOPS per m² of floorspace and FLOPS per unit cost, allowed scaling up to very high performance. With so many nodes, component failures were inevitable. The system was able to electrically isolate faulty components, down to a granularity of half a rack (512 compute nodes), to allow the machine to continue to run.

Each Blue Gene/L node was attached to three parallel communications networks: a 3D toroidal network for peer-to-peer communication between compute nodes, a collective network for collective communication (broadcasts and reduce operations), and a global interrupt network for fast barriers. The I/O nodes, which run the Linux operating system, provided communication to storage and external hosts via an Ethernet network. The I/O nodes handled filesystem operations on behalf of the compute nodes. Finally, a separate and private Ethernet network provided access to any node for configuration, booting and diagnostics. To allow multiple programs to run concurrently, a Blue Gene/L system could be partitioned into electronically isolated sets of nodes. The number of nodes in a partition had to be a positive integer power of 2, with at least 2⁵ = 32 nodes. To run a program on Blue Gene/L, a partition of the computer was first to be reserved. The program was then loaded and run on all the nodes within the partition, and no other program could access nodes within the partition while it was in use. Upon completion, the partition nodes were released for future programs to use.

Blue Gene/L compute nodes used a minimal operating system supporting a single user program. Only a subset of POSIX calls was supported, and only one process could run at a time on node in co-processor mode—or one process per CPU in virtual mode. Programmers needed to implement green threads in order to simulate local concurrency. Application development was usually performed in C, C++, or Fortran using MPI for communication. However, some scripting languages such as Ruby and Python have been ported to the compute nodes.

IBM has published BlueMatter, the application developed to exercise Blue Gene/L, as open source here. This serves to document how the torus and collective interfaces were used by applications, and may serve as a base for others to exercise the current generation of supercomputers.

Blue Gene/P

A Blue Gene/P node card

A schematic overview of a Blue Gene/P supercomputer

In June 2007, IBM unveiled Blue Gene/P, the second generation of the Blue Gene series of supercomputers and designed through a collaboration that included IBM, LLNL, and Argonne National Laboratory's Leadership Computing Facility.

Design

The design of Blue Gene/P is a technology evolution from Blue Gene/L. Each Blue Gene/P Compute chip contains four PowerPC 450 processor cores, running at 850 MHz. The cores are cache coherent and the chip can operate as a 4-way symmetric multiprocessor (SMP). The memory subsystem on the chip consists of small private L2 caches, a central shared 8 MB L3 cache, and dual DDR2 memory controllers. The chip also integrates the logic for node-to-node communication, using the same network topologies as Blue Gene/L, but at more than twice the bandwidth. A compute card contains a Blue Gene/P chip with 2 or 4 GB DRAM, comprising a "compute node". A single compute node has a peak performance of 13.6 GFLOPS. 32 Compute cards are plugged into an air-cooled node board. A rack contains 32 node boards (thus 1024 nodes, 4096 processor cores). By using many small, low-power, densely packaged chips, Blue Gene/P exceeded the power efficiency of other supercomputers of its generation, and at 371 MFLOPS/W Blue Gene/P installations ranked at or near the top of the Green500 lists in 2007-2008.

Installations

The following is an incomplete list of Blue Gene/P installations. Per November 2009, the TOP500 list contained 15 Blue Gene/P installations of 2-racks (2048 nodes, 8192 processor cores, 23.86 TFLOPS Linpack) and larger.

• On November 12, 2007, the first Blue Gene/P installation, JUGENE, with 16 racks (16,384 nodes, 65,536 processors) was running at Forschungszentrum Jülich in Germany with a performance of 167 TFLOPS. When inaugurated it was the fastest supercomputer in Europe and the sixth fastest in the world. In 2009, JUGENE was upgraded to 72 racks (73,728 nodes, 294,912 processor cores) with 144 terabytes of memory and 6 petabytes of storage, and achieved a peak performance of 1 PetaFLOPS. This configuration incorporated new air-to-water heat exchangers between the racks, reducing the cooling cost substantially. JUGENE was shut down in July 2012 and replaced by the Blue Gene/Q system JUQUEEN.
• The 40-rack (40960 nodes, 163840 processor cores) "Intrepid" system at Argonne National Laboratory was ranked #3 on the June 2008 Top 500 list. The Intrepid system is one of the major resources of the INCITE program, in which processor hours are awarded to "grand challenge" science and engineering projects in a peer-reviewed competition.
• Lawrence Livermore National Laboratory installed a 36-rack Blue Gene/P installation, "Dawn", in 2009.
• The King Abdullah University of Science and Technology (KAUST) installed a 16-rack Blue Gene/P installation, "Shaheen", in 2009.
• In 2012, a 6-rack Blue Gene/P was installed at Rice University and will be jointly administered with the University of São Paulo.
• A 2.5 rack Blue Gene/P system is the central processor for the Low Frequency Array for Radio astronomy (LOFAR) project in the Netherlands and surrounding European countries. This application uses the streaming data capabilities of the machine.
• A 2-rack Blue Gene/P was installed in September 2008 in Sofia, Bulgaria, and is operated by the Bulgarian Academy of Sciences and Sofia University.
• In 2010, a 2-rack (8192-core) Blue Gene/P was installed at the University of Melbourne for the Victorian Life Sciences Computation Initiative.
• In 2011, a 2-rack Blue Gene/P was installed at University of Canterbury in Christchurch, New Zealand.
• In 2012, a 2-rack Blue Gene/P was installed at Rutgers University in Piscataway, New Jersey. It was dubbed "Excalibur" as an homage to the Rutgers mascot, the Scarlet Knight.
• In 2008, a 1-rack (1024 nodes) Blue Gene/P with 180 TB of storage was installed at the University of Rochester in Rochester, New York.
• The first Blue Gene/P in the ASEAN region was installed in 2010 at the Universiti of Brunei Darussalam’s research centre, the UBD-IBM Centre. The installation has prompted research collaboration between the university and IBM research on climate modeling that will investigate the impact of climate change on flood forecasting, crop yields, renewable energy and the health of rainforests in the region among others.
• In 2013, a 1-rack Blue Gene/P was donated to the Department of Science and Technology for weather forecasts, disaster management, precision agriculture, and health it is housed in the National Computer Center, Diliman, Quezon City, under the auspices of Philippine Genome Center (PGC) Core Facility for Bioinformatics (CFB) at UP Diliman, Quezon City.

Applications

• Veselin Topalov, the challenger to the World Chess Champion title in 2010, confirmed in an interview that he had used a Blue Gene/P supercomputer during his preparation for the match.
• The Blue Gene/P computer has been used to simulate approximately one percent of a human cerebral cortex, containing 1.6 billion neurons with approximately 9 trillion connections.
• The IBM Kittyhawk project team has ported Linux to the compute nodes and demonstrated generic Web 2.0 workloads running at scale on a Blue Gene/P. Their paper, published in the ACM Operating Systems Review, describes a kernel driver that tunnels Ethernet over the tree network, which results in all-to-all TCP/IP connectivity. Running standard Linux software like MySQL, their performance results on SpecJBB rank among the highest on record.
• In 2011, a Rutgers University / IBM / University of Texas team linked the KAUST Shaheen installation together with a Blue Gene/P installation at the IBM Watson Research Center into a "federated high performance computing cloud", winning the IEEE SCALE 2011 challenge with an oil reservoir optimization application.

Blue Gene/Q

The IBM Blue Gene/Q installed at Argonne National Laboratory, near Chicago, Illinois.

The third supercomputer design in the Blue Gene series, Blue Gene/Q has a peak performance of 20 Petaflops, reaching LINPACK benchmarks performance of 17 Petaflops. Blue Gene/Q continues to expand and enhance the Blue Gene/L and /P architectures.

Design

The Blue Gene/Q Compute chip is an 18 core chip. The 64-bit A2 processor cores are 4-way simultaneously multithreaded, and run at 1.6 GHz. Each processor core has a SIMD Quad-vector double precision floating point unit (IBM QPX). 16 Processor cores are used for computing, and a 17th core for operating system assist functions such as interrupts, asynchronous I/O, MPI pacing and RAS. The 18th core is used as a redundant spare, used to increase manufacturing yield. The spared-out core is shut down in functional operation. The processor cores are linked by a crossbar switch to a 32 MB eDRAM L2 cache, operating at half core speed. The L2 cache is multi-versioned, supporting transactional memory and speculative execution, and has hardware support for atomic operations. L2 cache misses are handled by two built-in DDR3 memory controllers running at 1.33 GHz. The chip also integrates logic for chip-to-chip communications in a 5D torus configuration, with 2GB/s chip-to-chip links. The Blue Gene/Q chip is manufactured on IBM's copper SOI process at 45 nm. It delivers a peak performance of 204.8 GFLOPS at 1.6 GHz, drawing about 55 watts. The chip measures 19×19 mm (359.5 mm²) and comprises 1.47 billion transistors. The chip is mounted on a compute card along with 16 GB DDR3 DRAM (i.e., 1 GB for each user processor core).

A Q32 compute drawer contains 32 compute cards, each water cooled. A "midplane" (crate) contains 16 Q32 compute drawers for a total of 512 compute nodes, electrically interconnected in a 5D torus configuration (4x4x4x4x2). Beyond the midplane level, all connections are optical. Racks have two midplanes, thus 32 compute drawers, for a total of 1024 compute nodes, 16,384 user cores and 16 TB RAM.

Separate I/O drawers, placed at the top of a rack or in a separate rack, are air cooled and contain 8 compute cards and 8 PCIe expansion slots for InfiniBand or 10 Gigabit Ethernet networking.

Performance

At the time of the Blue Gene/Q system announcement in November 2011, an initial 4-rack Blue Gene/Q system (4096 nodes, 65536 user processor cores) achieved #17 in the TOP500 list with 677.1 TeraFLOPS Linpack, outperforming the original 2007 104-rack BlueGene/L installation described above. The same 4-rack system achieved the top position in the Graph500 list with over 250 GTEPS (giga traversed edges per second). Blue Gene/Q systems also topped the Green500 list of most energy efficient supercomputers with up to 2.1 GFLOPS/W.

In June 2012, Blue Gene/Q installations took the top positions in all three lists: TOP500, Graph500 and Green500.

Installations

The following is an incomplete list of Blue Gene/Q installations. Per June 2012, the TOP500 list contained 20 Blue Gene/Q installations of 1/2-rack (512 nodes, 8192 processor cores, 86.35 TFLOPS Linpack) and larger. At a (size-independent) power efficiency of about 2.1 GFLOPS/W, all these systems also populated the top of the June 2012 Green 500 list.

• A Blue Gene/Q system called Sequoia was delivered to the Lawrence Livermore National Laboratory (LLNL) beginning in 2011 and was fully deployed in June 2012. It is part of the Advanced Simulation and Computing Program running nuclear simulations and advanced scientific research. It consists of 96 racks (comprising 98,304 compute nodes with 1.6 million processor cores and 1.6 PB of memory) covering an area of about 3,000 square feet (280 m²). In June 2012, the system was ranked as the world's fastest supercomputer at 20.1 PFLOPS peak, 16.32 PFLOPS sustained (Linpack), drawing up to 7.9 megawatts of power. In June 2013, its performance is listed at 17.17 PFLOPS sustained (Linpack).
• A 10 PFLOPS (peak) Blue Gene/Q system called Mira was installed at Argonne National Laboratory in the Argonne Leadership Computing Facility in 2012. It consist of 48 racks (49,152 compute nodes), with 70 PB of disk storage (470 GB/s I/O bandwidth).
• JUQUEEN at the Forschungzentrum Jülich is a 28-rack Blue Gene/Q system, and was from June 2013 to November 2015 the highest ranked machine in Europe in the Top500.
• Vulcan at Lawrence Livermore National Laboratory (LLNL) is a 24-rack, 5 PFLOPS (peak), Blue Gene/Q system that was commissioned in 2012 and decommissioned in 2019. Vulcan served Lab-industry projects through Livermore's High Performance Computing (HPC) Innovation Center as well as academic collaborations in support of DOE/National Nuclear Security Administration (NNSA) missions.
• Fermi at the CINECA Supercomputing facility, Bologna, Italy, is a 10-rack, 2 PFLOPS (peak), Blue Gene/Q system.
• As part of DiRAC, the EPCC hosts a 6 rack (6144-node) Blue Gene/Q system at the University of Edinburgh
• A five rack Blue Gene/Q system with additional compute hardware called AMOS was installed at Rensselaer Polytechnic Institute in 2013. The system was rated at 1048.6 teraflops, the most powerful supercomputer at any private university, and third most powerful supercomputer among all universities in 2014.
• An 838 TFLOPS (peak) Blue Gene/Q system called Avoca was installed at the Victorian Life Sciences Computation Initiative in June, 2012. This system is part of a collaboration between IBM and VLSCI, with the aims of improving diagnostics, finding new drug targets, refining treatments and furthering our understanding of diseases. The system consists of 4 racks, with 350 TB of storage, 65,536 cores, 64 TB RAM.
• A 209 TFLOPS (peak) Blue Gene/Q system was installed at the University of Rochester in July, 2012. This system is part of the Health Sciences Center for Computational Innovation, which is dedicated to the application of high-performance computing to research programs in the health sciences. The system consists of a single rack (1,024 compute nodes) with 400 TB of high-performance storage.
• A 209 TFLOPS peak (172 TFLOPS LINPACK) Blue Gene/Q system called Lemanicus was installed at the EPFL in March 2013. This system belongs to the Center for Advanced Modeling Science CADMOS which is a collaboration between the three main research institutions on the shore of the Lake Geneva in the French speaking part of Switzerland : University of Lausanne, University of Geneva and EPFL. The system consists of a single rack (1,024 compute nodes) with 2.1 PB of IBM GPFS-GSS storage.
• A half-rack Blue Gene/Q system, with about 100 TFLOPS (peak), called Cumulus was installed at A*STAR Computational Resource Centre, Singapore, at early 2011.

Applications

Record-breaking science applications have been run on the BG/Q, the first to cross 10 petaflops of sustained performance. The cosmology simulation framework HACC achieved almost 14 petaflops with a 3.6 trillion particle benchmark run, while the Cardioid code, which models the electrophysiology of the human heart, achieved nearly 12 petaflops with a near real-time simulation, both on Sequoia. A fully compressible flow solver has also achieved 14.4 PFLOP/s (originally 11 PFLOP/s) on Sequoia, 72% of the machine's nominal peak performance.

Deep Blue (chess computer)

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer) 

 

Deep Blue

One of two racks of Deep Blue, at the Computer History Museum
Active	1995 (prototype) 1996 (release) 1997 (upgrade to Deep Blue II)
Architecture	IBM RS/6000 SP platform (32 nodes): 1996: 32 POWER2 (120 MHz) CPUs + 512 VLSI chess chips 1997: 32 P2SC (200 MHz) + 512 VLSI chess chips
Operating system	IBM AIX
Space	2 racks
Speed	11.38 GFLOPS (1997)
Purpose	Chess playing

This article is part of the series on
Chess programming
Board representations
0x88 Bitboards
Chess computers
Belle Deep Blue
Chess engines
Fritz GNU Chess Leela Chess Zero Stockfish

Deep Blue was a chess-playing computer developed by IBM. It was the first computer to win both a chess game and a chess match against a reigning world champion under regular time controls.

Development for Deep Blue began in 1985 with the ChipTest project at Carnegie Mellon University; Grandmaster Joel Benjamin was part of the development team. IBM hired the development team when the project was briefly given the name Deep Thought. In 1989, it was renamed Deep Blue.

Deep Blue won its first game against world champion Garry Kasparov in game one of a six-game match on 10 February 1996. However, Kasparov won three and drew two of the following five games, defeating Deep Blue by a score of 4–2. Deep Blue was heavily upgraded before playing against Kasparov again in May 1997. Deep Blue won game six, thereby winning the six-game rematch 3½–2½ and becoming the first computer system to defeat a reigning world champion in a match under standard chess tournament time controls. However, Kasparov accused IBM of cheating.

Origins

The project started under the name ChipTest at Carnegie Mellon University by Feng-hsiung Hsu and was followed by ChipTest’s successor, Deep Thought. After graduating the university, Hsu, Thomas Anantharaman, and Murray Campbell were asked by IBM Research to continue their project to build a chess machine that could defeat a world champion. Hsu and Campbell joined IBM in fall 1989, with Anantharaman following later. Anantharaman subsequently left IBM for Wall Street and Arthur Joseph Hoane joined the team to perform programming tasks. Jerry Brody, a long-time employee of IBM Research, was recruited to the team in 1990. The team was first managed by Randy Moulic, followed by Chung-Jen (C J) Tan.

After Deep Thought's 1989 match against Kasparov, IBM held a contest to rename the chess machine: the winning name was "Deep Blue", a play on IBM's nickname, "Big Blue". After a scaled-down version of Deep Blue—Deep Blue Jr.—played Grandmaster Joel Benjamin, Hsu and Campbell decided that Benjamin was the expert they were looking for to develop Deep Blue's opening book, and Benjamin was signed by IBM Research to assist with the preparations for Deep Blue's matches against Garry Kasparov.

In 1995, "Deep Blue prototype" played in the 8th World Computer Chess Championship. The Deep Blue prototype played Wchess to a draw. In round 5, Deep Blue prototype played as White and lost to Fritz.

Design

Deep Blue used custom VLSI chips to execute the alpha-beta search algorithm in parallel, an example of GOFAI (Good Old-Fashioned Artificial Intelligence).

The system derived its playing strength mainly from brute force computing power. It was a massively parallel, RS/6000 SP Thin P2SC-based system with 30 nodes, with each node containing a 120 MHz P2SC microprocessor enhanced with 480 special purpose VLSI chess chips. Its chess playing program was written in C and ran under the AIX operating system. It was capable of evaluating 200 million positions per second, twice as fast as the 1996 version. In 1997 Deep Blue was upgraded again. In June 1997, Deep Blue was the 259th most powerful supercomputer according to the TOP500 list, achieving 11.38 GFLOPS on the High-Performance LINPACK benchmark.

Deep Blue's evaluation function was initially written in a generalized form, with many to-be-determined parameters (e.g., how important is a safe king position compared to a space advantage in the center, etc.). The system determined the optimal values for these parameters by analyzing thousands of master games. The evaluation function had been split into 8,000 parts, many of them designed for special positions. In the opening book there were over 4,000 positions and 700,000 grandmaster games. The endgame database contained many six-piece endgames and five or fewer piece positions. Before the second match, the program's chess knowledge was fine-tuned by grandmaster Joel Benjamin. The opening library was provided by grandmasters Miguel Illescas, John Fedorowicz, and Nick de Firmian. When Kasparov requested that he be allowed to study other games that Deep Blue had played so as to better understand his opponent, IBM refused. However, Kasparov studied many popular PC games to become familiar with computer gameplay in general.

Deep Blue takes an approach using the opening information in its database. It creates an additional database called the “extended book.” The extended book summarizes previous Grandmaster games in any of the several million opening positions in its game database. The system can combine its big searching ability (200 million chess positions per second) with the summary information in the extended book to select opening moves.

Deep Blue versus Kasparov

Deep Blue and Kasparov played each other on two occasions. The first match began on 10 February 1996, in which Deep Blue became the first machine to win a chess game against a reigning world champion (Garry Kasparov) under regular time controls. However, Kasparov won three and drew two of the following five games, beating Deep Blue by a score of 4–2 (wins count as 1 point, draws count as a ½ point). The match concluded on 17 February 1996.

After the match, Deep Blue was upgraded (unofficially nicknamed "Deeper Blue") and played Kasparov again in May 1997, winning the six-game rematch 3½–2½, ending on 11 May. Deep Blue won the deciding game after Kasparov made a mistake in the opening and became the first computer system to defeat a reigning world champion in a match under standard chess tournament time controls.

The Deep Blue chess computer that defeated Kasparov in 1997 would typically search to a depth of between six and eight moves to twenty or even more moves in some situations. David Levy and Monty Newborn estimate that one additional ply (half-move) increases the playing strength between 50 and 70 Elo points.

Kasparov in 1985

Writer Nate Silver suggests that a bug in Deep Blue's software led to a seemingly random move (the 44th in the first game of the second match) which Kasparov misattributed to "superior intelligence". Subsequently, Kasparov experienced a decline in performance due to anxiety in the following game, though he rejects this interpretation.

After the loss, Kasparov said that he sometimes saw deep intelligence and creativity in the machine's moves, suggesting that during the second game, human chess players had intervened on behalf of the machine, which would be a violation of the rules. IBM denied that it cheated, saying the only human intervention occurred between games. Kasparov demanded a rematch, but IBM had dismantled Deep Blue after its victory and refused the rematch. The rules allowed the developers to modify the program between games, an opportunity they said they used to shore up weaknesses in the computer's play that were revealed during the course of the match. Kasparov requested printouts of the machine's log files, but IBM refused, although the company later published the logs on the Internet.

Aftermath

Computer scientists believed that playing chess was a good measurement for the effectiveness of artificial intelligence, and by beating a world champion chess player, IBM showed that they had made significant progress. Kasparov called Deep Blue an "alien opponent" but later stated that "It was as intelligent as your alarm clock". According to Martin Amis, two grandmasters who played Deep Blue agreed with each other that "It's like a wall coming at you".

In 2003 a documentary film—Game Over: Kasparov and the Machine—was made that explored these claims. It interviewed some people who suggest that Deep Blue's victory was a ploy by IBM to boost its stock value.

One of the cultural impacts of Deep Blue was the creation of a new game called Arimaa, which was designed to be much more difficult for computers than chess. Computers proved capable of defeating strong Arimaa players in 2015.

One of the two racks that made up Deep Blue is held by the National Museum of American History, having previously been displayed in an exhibit about the Information Age; the other rack is displayed at the Computer History Museum in the Revolution exhibit's "Artificial Intelligence and Robotics" gallery. Deep Blue was mistakenly reported to be sold to United Airlines as it was confused with other RS6000/SP2 systems systems.

Feng-hsiung Hsu later wrote in his book Behind Deep Blue that he had the rights to use the Deep Blue design to build a bigger machine independently of IBM to take Kasparov's rematch offer, but Kasparov refused a rematch.

Deep Blue, with its capability of evaluating 200 million positions per second, was the first and fastest computer to face a world chess champion. Today, in computer-chess research and matches of world-class players against computers, the focus of play has shifted to software chess programs, rather than using dedicated chess hardware. Modern chess programs like Houdini, Rybka, Deep Fritz or Deep Junior are more efficient than the programs during Deep Blue's era. In a November 2006 match between Deep Fritz and world chess champion Vladimir Kramnik, the program ran on a computer system containing a dual-core Intel Xeon 5160 CPU, capable of evaluating only 8 million positions per second, but searching to an average depth of 17 to 18 plies in the middlegame thanks to heuristics; it won 4–2.

IBM Deep Thunder

From Wikipedia, the free encyclopedia

This article needs to be updated. Please update this article to reflect recent events or newly available information. (December 2016)

Deep Thunder is a research project by IBM that aims to improve short-term local weather forecasting through the use of high-performance computing. It is part of IBM's Deep Computing initiative that also produced the Deep Blue chess computer.

Deep Thunder is intended to provide local, high-resolution weather predictions customized to weather-sensitive specific business operations. For example, it could be used to predict the wind velocity at an Olympic diving platform, destructive thunderstorms, and combined with other physical models to predict where there will be flooding, damaged power lines and algal blooms. The project is now headquartered at IBM's Thomas J. Watson Research Center in Yorktown Heights, New York.

History

The Deep Thunder project is headed by Lloyd Treinish, who joined IBM in 1990, after working for 12 years at NASA's Goddard Space Flight Center.

The project began in 1995 as an outgrowth from a project designed to help provide accurate weather forecasts for the 1996 Atlanta Olympic Games. In collaboration with the National Oceanic and Atmospheric Administration, IBM scientists built one of the first parallel processing supercomputers to be used for weather modeling, based on the IBM RS/6000 SP. It was installed at the National Weather Service office in Peachtree City, Georgia, in 1996, where it ran for several months and produced multiple forecasts daily. After a few years of development, the team set up an implementation in New York City in 2001 to test the project. The group is currently working on establishing the Rio de Janeiro operations center.

The name Deep Thunder arose after the IBM Deep Blue system played and defeated the world chess champion Garry Kasparov in May, 1997. In the following November, a journalist used the name Deep Thunder in an article, which stuck with the developers. Current members of Deep Thunder are Lloyd Treinish, Anthony Praino, Campbell Watson and Mukul Tewari.

Technology

Deep Thunder uses a 3D telescoping grid where data from one model feeds into another and is verified with historical data. For example, they start off with a global model from NOAA, and as they zoom in the resolution decreases exponentially, down to models with resolutions of 1 kilometer, and sometimes as small as 1 meter. Using this method, IBM is able to cut down on the amount of processing required. IBM uses many sources of data to feed Deep Thunder, including public satellite sources, and many other private sources, as well as whatever local sensors and data a location may have.

The Watson computer system will be used to generate the Deep Thunder weather forecasts. Input data will be collected from over 200,000 Weather Underground personal weather stations, weather satellite data, smartphone barometer and data from other sources.

Applications

Utility Companies

IBM worked with a North American utility company that has over 90,000 poles, wires, and transformers to develop a prediction service that can pinpoint where incoming storms will bring down trees and power lines. The service can be used to call in the needed number of repair crews and station them near where the damage will occur, drastically decreasing downtime.

Agriculture

Deep Thunder could be used to determine optimal times to plant, irrigate, and harvest crops, based on the dynamic environmental conditions of individual farm locations. Precision agriculture using Deep Thunder could lead to better price points for crops by saving water, by allocating labor more effectively, and by improving supply chain efficiency. Using these methods of predictive weather farming, crop losses from weather can be cut by over 25%. The island nation of Brunei is working with Deep Thunder to develop precision weather forecast models to increase rice production.

In The Olympics

Deep Thunder was used during the 1996 Atlanta Olympic Games to accurately predict that there would be no rain during the closing ceremony. It is also intended to be used for the 2016 Summer Olympics in Rio de Janeiro.

The Jefferson Project

The Jefferson Project at Lake George (New York) is a global example of sustained protection of freshwater ecosystems. The project uses Deep Thunder to predict the weather at an unprecedented 333 m resolution, simulating complex airflow patterns over the lake that are crucial to lake currents and nutrient cycling.

New York

New York City was the first city to test a full scale implementation of Deep Thunder. IBM is experimenting with using a mobile app to distribute location-specific predictions and to issue alerts. Data from the app can be used by maintenance crews to determine if wind levels are too high to work, or it could be used to get a weather forecast at a certain address.

Rio de Janeiro

IBM is currently expanding the project to Rio de Janeiro in order to predict floods, and anticipate where storms might trigger mudslides. The city is collaborating with IBM on a multimillion-dollar plan to improve emergency responsiveness, by providing comprehensive information about rainfall estimates, wind speed and direction, probabilities of landslides and floods to responders. It is expected to be utilized during the 2014 FIFA World Cup and 2016 Summer Olympics.

Dublin, Ireland

The Dublin City Council is working with IBM to help make Dublin, Ireland, the third city in the world to implement the Deep Thunder forecasting model in an attempt to predict and issue warnings about incoming flash floods that are impacting businesses and homeowners.

 

Watson (computer)

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Watson_(computer) 

Watson's avatar, inspired by the IBM "Smarter Planet" logo

Watson is a question-answering computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named after IBM's founder and first CEO, industrialist Thomas J. Watson.

The computer system was initially developed to answer questions on the quiz show Jeopardy! and, in 2011, the Watson computer system competed on Jeopardy! against champions Brad Rutter and Ken Jennings, winning the first place prize of $1 million.

In February 2013, IBM announced that Watson software system's first commercial application would be for utilization management decisions in lung cancer treatment at Memorial Sloan Kettering Cancer Center, New York City, in conjunction with WellPoint (now Anthem). In 2013, Manoj Saxena, IBM Watson's business chief said that 90% of nurses in the field who use Watson now follow its guidance.

Description

The high-level architecture of IBM's DeepQA used in Watson

Watson was created as a question answering (QA) computing system that IBM built to apply advanced natural language processing, information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering.

When created, IBM stated that, "more than 100 different techniques are used to analyze natural language, identify sources, find and generate hypotheses, find and score evidence, and merge and rank hypotheses."

In recent years, the Watson capabilities have been extended and the way in which Watson works has been changed to take advantage of new deployment models (Watson on IBM Cloud) and evolved machine learning capabilities and optimised hardware available to developers and researchers. It is no longer purely a question answering (QA) computing system designed from Q&A pairs but can now 'see', 'hear', 'read', 'talk', 'taste', 'interpret', 'learn' and 'recommend'.

Software

Watson uses IBM's DeepQA software and the Apache UIMA (Unstructured Information Management Architecture) framework implementation. The system was written in various languages, including Java, C++, and Prolog, and runs on the SUSE Linux Enterprise Server 11 operating system using the Apache Hadoop framework to provide distributed computing.

Hardware

The system is workload-optimized, integrating massively parallel POWER7 processors and built on IBM's DeepQA technology, which it uses to generate hypotheses, gather massive evidence, and analyze data. Watson employs a cluster of ninety IBM Power 750 servers, each of which uses a 3.5 GHz POWER7 eight-core processor, with four threads per core. In total, the system has 2,880 POWER7 processor threads and 16 terabytes of RAM.

According to John Rennie, Watson can process 500 gigabytes, the equivalent of a million books, per second. IBM's master inventor and senior consultant, Tony Pearson, estimated Watson's hardware cost at about three million dollars. Its Linpack performance stands at 80 TeraFLOPs, which is about half as fast as the cut-off line for the Top 500 Supercomputers list. According to Rennie, all content was stored in Watson's RAM for the Jeopardy game because data stored on hard drives would be too slow to be competitive with human Jeopardy champions.

Data

The sources of information for Watson include encyclopedias, dictionaries, thesauri, newswire articles and literary works. Watson also used databases, taxonomies and ontologies including DBPedia, WordNet and Yago. The IBM team provided Watson with millions of documents, including dictionaries, encyclopedias and other reference material that it could use to build its knowledge.peration

Watson parses questions into different keywords and sentence fragments in order to find statistically related phrases. Watson's main innovation was not in the creation of a new algorithm for this operation but rather its ability to quickly execute hundreds of proven language analysis algorithms simultaneously. The more algorithms that find the same answer independently, the more likely Watson is to be correct. Once Watson has a small number of potential solutions, it is able to check against its database to ascertain whether the solution makes sense or not.

Comparison with human players

Ken Jennings, Watson, and Brad Rutter in their Jeopardy! exhibition match.

Watson's basic working principle is to parse keywords in a clue while searching for related terms as responses. This gives Watson some advantages and disadvantages compared with human Jeopardy! players. Watson has deficiencies in understanding the contexts of the clues. As a result, human players usually generate responses faster than Watson, especially to short clues. Watson's programming prevents it from using the popular tactic of buzzing before it is sure of its response. Watson has consistently better reaction time on the buzzer once it has generated a response, and is immune to human players' psychological tactics, such as jumping between categories on every clue.

In a sequence of 20 mock games of Jeopardy, human participants were able to use the average six to seven seconds that Watson needed to hear the clue and decide whether to signal for responding. During that time, Watson also has to evaluate the response and determine whether it is sufficiently confident in the result to signal. Part of the system used to win the Jeopardy! contest was the electronic circuitry that receives the "ready" signal and then examined whether Watson's confidence level was great enough to activate the buzzer. Given the speed of this circuitry compared to the speed of human reaction times, Watson's reaction time was faster than the human contestants except when the human anticipated (instead of reacted to) the ready signal. After signaling, Watson speaks with an electronic voice and gives the responses in Jeopardy!'s question format. Watson's voice was synthesized from recordings that actor Jeff Woodman made for an IBM text-to-speech program in 2004.

The Jeopardy! staff used different means to notify Watson and the human players when to buzz, which was critical in many rounds. The humans were notified by a light, which took them tenths of a second to perceive. Watson was notified by an electronic signal and could activate the buzzer within about eight milliseconds. The humans tried to compensate for the perception delay by anticipating the light, but the variation in the anticipation time was generally too great to fall within Watson's response time. Watson did not attempt to anticipate the notification signal.

History

Development

Since Deep Blue's victory over Garry Kasparov in chess in 1997, IBM had been on the hunt for a new challenge. In 2004, IBM Research manager Charles Lickel, over dinner with coworkers, noticed that the restaurant they were in had fallen silent. He soon discovered the cause of this evening hiatus: Ken Jennings, who was then in the middle of his successful 74-game run on Jeopardy!. Nearly the entire restaurant had piled toward the televisions, mid-meal, to watch Jeopardy!. Intrigued by the quiz show as a possible challenge for IBM, Lickel passed the idea on, and in 2005, IBM Research executive Paul Horn supported Lickel, pushing for someone in his department to take up the challenge of playing Jeopardy! with an IBM system. Though he initially had trouble finding any research staff willing to take on what looked to be a much more complex challenge than the wordless game of chess, eventually David Ferrucci took him up on the offer. In competitions managed by the United States government, Watson's predecessor, a system named Piquant, was usually able to respond correctly to only about 35% of clues and often required several minutes to respond. To compete successfully on Jeopardy!, Watson would need to respond in no more than a few seconds, and at that time, the problems posed by the game show were deemed to be impossible to solve.

In initial tests run during 2006 by David Ferrucci, the senior manager of IBM's Semantic Analysis and Integration department, Watson was given 500 clues from past Jeopardy! programs. While the best real-life competitors buzzed in half the time and responded correctly to as many as 95% of clues, Watson's first pass could get only about 15% correct. During 2007, the IBM team was given three to five years and a staff of 15 people to solve the problems. John E. Kelly III succeeded Paul Horn as head of IBM Research in 2007. InformationWeek described Kelly as "the father of Watson" and credited him for encouraging the system to compete against humans on Jeopardy!. By 2008, the developers had advanced Watson such that it could compete with Jeopardy! champions. By February 2010, Watson could beat human Jeopardy! contestants on a regular basis.

During the game, Watson had access to 200 million pages of structured and unstructured content consuming four terabytes of disk storage including the full text of the 2011 edition of Wikipedia, but was not connected to the Internet. For each clue, Watson's three most probable responses were displayed on the television screen. Watson consistently outperformed its human opponents on the game's signaling device, but had trouble in a few categories, notably those having short clues containing only a few words.

Although the system is primarily an IBM effort, Watson's development involved faculty and graduate students from Rensselaer Polytechnic Institute, Carnegie Mellon University, University of Massachusetts Amherst, the University of Southern California's Information Sciences Institute, the University of Texas at Austin, the Massachusetts Institute of Technology, and the University of Trento, as well as students from New York Medical College.

Jeopardy!

Preparation

Watson demo at an IBM booth at a trade show

In 2008, IBM representatives communicated with Jeopardy! executive producer Harry Friedman about the possibility of having Watson compete against Ken Jennings and Brad Rutter, two of the most successful contestants on the show, and the program's producers agreed. Watson's differences with human players had generated conflicts between IBM and Jeopardy! staff during the planning of the competition. IBM repeatedly expressed concerns that the show's writers would exploit Watson's cognitive deficiencies when writing the clues, thereby turning the game into a Turing test. To alleviate that claim, a third party randomly picked the clues from previously written shows that were never broadcast. Jeopardy! staff also showed concerns over Watson's reaction time on the buzzer. Originally Watson signalled electronically, but show staff requested that it press a button physically, as the human contestants would. Even with a robotic "finger" pressing the buzzer, Watson remained faster than its human competitors. Ken Jennings noted, "If you're trying to win on the show, the buzzer is all", and that Watson "can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard." Stephen Baker, a journalist who recorded Watson's development in his book Final Jeopardy, reported that the conflict between IBM and Jeopardy! became so serious in May 2010 that the competition was almost canceled. As part of the preparation, IBM constructed a mock set in a conference room at one of its technology sites to model the one used on Jeopardy!. Human players, including former Jeopardy! contestants, also participated in mock games against Watson with Todd Alan Crain of The Onion playing host. About 100 test matches were conducted with Watson winning 65% of the games.

To provide a physical presence in the televised games, Watson was represented by an "avatar" of a globe, inspired by the IBM "smarter planet" symbol. Jennings described the computer's avatar as a "glowing blue ball criss-crossed by 'threads' of thought—42 threads, to be precise", and stated that the number of thought threads in the avatar was an in-joke referencing the significance of the number 42 in Douglas Adams' Hitchhiker's Guide to the Galaxy. Joshua Davis, the artist who designed the avatar for the project, explained to Stephen Baker that there are 36 triggerable states that Watson was able to use throughout the game to show its confidence in responding to a clue correctly; he had hoped to be able to find forty-two, to add another level to the Hitchhiker's Guide reference, but he was unable to pinpoint enough game states.

A practice match was recorded on January 13, 2011, and the official matches were recorded on January 14, 2011. All participants maintained secrecy about the outcome until the match was broadcast in February.

Practice match

In a practice match before the press on January 13, 2011, Watson won a 15-question round against Ken Jennings and Brad Rutter with a score of $4,400 to Jennings's $3,400 and Rutter's $1,200, though Jennings and Watson were tied before the final $1,000 question. None of the three players responded incorrectly to a clue.

First match

The first round was broadcast February 14, 2011, and the second round, on February 15, 2011. The right to choose the first category had been determined by a draw won by Rutter. Watson, represented by a computer monitor display and artificial voice, responded correctly to the second clue and then selected the fourth clue of the first category, a deliberate strategy to find the Daily Double as quickly as possible. Watson's guess at the Daily Double location was correct. At the end of the first round, Watson was tied with Rutter at $5,000; Jennings had $2,000.

Watson's performance was characterized by some quirks. In one instance, Watson repeated a reworded version of an incorrect response offered by Jennings. (Jennings said "What are the '20s?" in reference to the 1920s. Then Watson said "What is 1920s?") Because Watson could not recognize other contestants' responses, it did not know that Jennings had already given the same response. In another instance, Watson was initially given credit for a response of "What is a leg?" after Jennings incorrectly responded "What is: he only had one hand?" to a clue about George Eyser (the correct response was, "What is: he's missing a leg?"). Because Watson, unlike a human, could not have been responding to Jennings's mistake, it was decided that this response was incorrect. The broadcast version of the episode was edited to omit Trebek's original acceptance of Watson's response. Watson also demonstrated complex wagering strategies on the Daily Doubles, with one bet at $6,435 and another at $1,246. Gerald Tesauro, one of the IBM researchers who worked on Watson, explained that Watson's wagers were based on its confidence level for the category and a complex regression model called the Game State Evaluator.

Watson took a commanding lead in Double Jeopardy!, correctly responding to both Daily Doubles. Watson responded to the second Daily Double correctly with a 32% confidence score.

However during the Final Jeopardy! round, Watson was the only contestant to miss the clue in the category U.S. Cities ("Its largest airport was named for a World War II hero; its second largest, for a World War II battle"). Rutter and Jennings gave the correct response of Chicago, but Watson's response was "What is Toronto?????" with five question marks appended indicating a lack of confidence. Ferrucci offered reasons why Watson would appear to have guessed a Canadian city: categories only weakly suggest the type of response desired, the phrase "U.S. city" did not appear in the question, there are cities named Toronto in the U.S., and Toronto in Ontario has an American League baseball team. Dr. Chris Welty, who also worked on Watson, suggested that it may not have been able to correctly parse the second part of the clue, "its second largest, for a World War II battle" (which was not a standalone clause despite it following a semicolon, and required context to understand that it was referring to a second-largest airport). Eric Nyberg, a professor at Carnegie Mellon University and a member of the development team, stated that the error occurred because Watson does not possess the comparative knowledge to discard that potential response as not viable. Although not displayed to the audience as with non-Final Jeopardy! questions, Watson's second choice was Chicago. Both Toronto and Chicago were well below Watson's confidence threshold, at 14% and 11% respectively. Watson wagered only $947 on the question.

The game ended with Jennings with $4,800, Rutter with $10,400, and Watson with $35,734.

Second match

During the introduction, Trebek (a Canadian native) joked that he had learned Toronto was a U.S. city, and Watson's error in the first match prompted an IBM engineer to wear a Toronto Blue Jays jacket to the recording of the second match.

In the first round, Jennings was finally able to choose a Daily Double clue, while Watson responded to one Daily Double clue incorrectly for the first time in the Double Jeopardy! Round. After the first round, Watson placed second for the first time in the competition after Rutter and Jennings were briefly successful in increasing their dollar values before Watson could respond. Nonetheless, the final result ended with a victory for Watson with a score of $77,147, besting Jennings who scored $24,000 and Rutter who scored $21,600.

Final outcome

The prizes for the competition were $1 million for first place (Watson), $300,000 for second place (Jennings), and $200,000 for third place (Rutter). As promised, IBM donated 100% of Watson's winnings to charity, with 50% of those winnings going to World Vision and 50% going to World Community Grid. Similarly, Jennings and Rutter donated 50% of their winnings to their respective charities.

In acknowledgment of IBM and Watson's achievements, Jennings made an additional remark in his Final Jeopardy! response: "I for one welcome our new computer overlords", echoing a similar meme to the episode "Deep Space Homer" on The Simpsons, in which TV news presenter Kent Brockman speaks of welcoming "our new insect overlords". Jennings later wrote an article for Slate, in which he stated:

IBM has bragged to the media that Watson's question-answering skills are good for more than annoying Alex Trebek. The company sees a future in which fields like medical diagnosis, business analytics, and tech support are automated by question-answering software like Watson. Just as factory jobs were eliminated in the 20th century by new assembly-line robots, Brad and I were the first knowledge-industry workers put out of work by the new generation of 'thinking' machines. 'Quiz show contestant' may be the first job made redundant by Watson, but I'm sure it won't be the last.

Philosophy

Philosopher John Searle argues that Watson—despite impressive capabilities—cannot actually think. Drawing on his Chinese room thought experiment, Searle claims that Watson, like other computational machines, is capable only of manipulating symbols, but has no ability to understand the meaning of those symbols; however, Searle's experiment has its detractors.

Match against members of the United States Congress

On February 28, 2011, Watson played an untelevised exhibition match of Jeopardy! against members of the United States House of Representatives. In the first round, Rush D. Holt, Jr. (D-NJ, a former Jeopardy! contestant), who was challenging the computer with Bill Cassidy (R-LA, later Senator from Louisiana), led with Watson in second place. However, combining the scores between all matches, the final score was $40,300 for Watson and $30,000 for the congressional players combined.

IBM's Christopher Padilla said of the match, "The technology behind Watson represents a major advancement in computing. In the data-intensive environment of government, this type of technology can help organizations make better decisions and improve how government helps its citizens."

Current and future applications

According to IBM, "The goal is to have computers start to interact in natural human terms across a range of applications and processes, understanding the questions that humans ask and providing answers that humans can understand and justify." It has been suggested by Robert C. Weber, IBM's general counsel, that Watson may be used for legal research. The company also intends to use Watson in other information-intensive fields, such as telecommunications, financial services, and government.

Watson is based on commercially available IBM Power 750 servers that have been marketed since February 2010. IBM also intends to market the DeepQA software to large corporations, with a price in the millions of dollars, reflecting the $1 million needed to acquire a server that meets the minimum system requirement to operate Watson. IBM expects the price to decrease substantially within a decade as the technology improves.

Commentator Rick Merritt said that "there's another really important reason why it is strategic for IBM to be seen very broadly by the American public as a company that can tackle tough computer problems. A big slice of [IBM's profit] comes from selling to the U.S. government some of the biggest, most expensive systems in the world."

In 2013, it was reported that three companies were working with IBM to create apps embedded with Watson technology. Fluid is developing an app for retailers, one called "The North Face", which is designed to provide advice to online shoppers. Welltok is developing an app designed to give people advice on ways to engage in activities to improve their health. MD Buyline is developing an app for the purpose of advising medical institutions on equipment procurement decisions.

In November 2013, IBM announced it would make Watson's API available to software application providers, enabling them to build apps and services that are embedded in Watson's capabilities. To build out its base of partners who create applications on the Watson platform, IBM consults with a network of venture capital firms, which advise IBM on which of their portfolio companies may be a logical fit for what IBM calls the Watson Ecosystem. Thus far, roughly 800 organizations and individuals have signed up with IBM, with interest in creating applications that could use the Watson platform.

On January 30, 2013, it was announced that Rensselaer Polytechnic Institute would receive a successor version of Watson, which would be housed at the Institute's technology park and be available to researchers and students. By summer 2013, Rensselaer had become the first university to receive a Watson computer.

On February 6, 2014, it was reported that IBM plans to invest $100 million in a 10-year initiative to use Watson and other IBM technologies to help countries in Africa address development problems, beginning with healthcare and education.

On June 3, 2014, three new Watson Ecosystem partners were chosen from more than 400 business concepts submitted by teams spanning 18 industries from 43 countries. "These bright and enterprising organizations have discovered innovative ways to apply Watson that can deliver demonstrable business benefits", said Steve Gold, vice president, IBM Watson Group. The winners were Majestyk Apps with their adaptive educational platform, FANG (Friendly Anthropomorphic Networked Genome); Red Ant with their retail sales trainer; and GenieMD with their medical recommendation service.

On July 9, 2014, Genesys Telecommunications Laboratories announced plans to integrate Watson to improve their customer experience platform, citing the sheer volume of customer data to analyze is staggering.

Watson has been integrated with databases including Bon Appétit magazine to perform a recipe generating platform.

Watson is being used by Decibel, a music discovery startup, in its app MusicGeek which uses the supercomputer to provide music recommendations to its users. The use of the artificial intelligence of Watson has also been found in the hospitality industry. GoMoment uses Watson for its Rev1 app, which gives hotel staff a way to quickly respond to questions from guests. Arria NLG has built an app that helps energy companies stay within regulatory guidelines, making it easier for managers to make sense of thousands of pages of legal and technical jargon.

OmniEarth, Inc. uses Watson computer vision services to analyze satellite and aerial imagery, along with other municipal data, to infer water usage on a property-by-property basis, helping water districts in drought-stricken California improve water conservation efforts.

In September 2016, Condé Nast has started using IBM's Watson to help build and strategize social influencer campaigns for brands. Using software built by IBM and Influential, Condé Nast's clients will be able to know which influencer's demographics, personality traits and more best align with a marketer and the audience it is targeting.

In February 2017, Rare Carat, a New York City-based startup and e-commerce platform for buying diamonds and diamond rings, introduced an IBM Watson-powered artificial intelligence chatbot called "Rocky" to assist novice diamond buyers through the daunting process of purchasing a diamond. As part of the IBM Global Entrepreneur Program, Rare Carat received the assistance of IBM in the development of the Rocky Chat Bot. In May 2017, IBM partnered with the Pebble Beach Company to use Watson as a concierge. Watson's artificial intelligence was added to an app developed by Pebble Beach and was used to guide visitors around the resort. The mobile app was designed by IBM iX and hosted on the IBM Cloud. It uses Watson's Conversation applications programming interface.

In November 2017, in Mexico City, the Experience Voices of Another Time was opened at the National Museum of Anthropology using IBM Watson as an alternative to visiting a museum.

Healthcare

In healthcare, Watson's natural language, hypothesis generation, and evidence-based learning capabilities are being investigated to see how Watson may contribute to clinical decision support systems and the increase in artificial intelligence in healthcare for use by medical professionals. To aid physicians in the treatment of their patients, once a physician has posed a query to the system describing symptoms and other related factors, Watson first parses the input to identify the most important pieces of information; then mines patient data to find facts relevant to the patient's medical and hereditary history; then examines available data sources to form and test hypotheses; and finally provides a list of individualized, confidence-scored recommendations. The sources of data that Watson uses for analysis can include treatment guidelines, electronic medical record data, notes from healthcare providers, research materials, clinical studies, journal articles and patient information. Despite being developed and marketed as a "diagnosis and treatment advisor", Watson has never been actually involved in the medical diagnosis process, only in assisting with identifying treatment options for patients who have already been diagnosed.

In February 2011, it was announced that IBM would be partnering with Nuance Communications for a research project to develop a commercial product during the next 18 to 24 months, designed to exploit Watson's clinical decision support capabilities. Physicians at Columbia University would help to identify critical issues in the practice of medicine where the system's technology may be able to contribute, and physicians at the University of Maryland would work to identify the best way that a technology like Watson could interact with medical practitioners to provide the maximum assistance.

In September 2011, IBM and WellPoint (now Anthem) announced a partnership to utilize Watson's data crunching capability to help suggest treatment options to physicians. Then, in February 2013, IBM and WellPoint gave Watson its first commercial application, for utilization management decisions in lung cancer treatment at Memorial Sloan–Kettering Cancer Center.

IBM announced a partnership with Cleveland Clinic in October 2012. The company has sent Watson to the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, where it will increase its health expertise and assist medical professionals in treating patients. The medical facility will utilize Watson's ability to store and process large quantities of information to help speed up and increase the accuracy of the treatment process. "Cleveland Clinic's collaboration with IBM is exciting because it offers us the opportunity to teach Watson to 'think' in ways that have the potential to make it a powerful tool in medicine", said C. Martin Harris, MD, chief information officer of Cleveland Clinic.

In 2013, IBM and MD Anderson Cancer Center began a pilot program to further the center's "mission to eradicate cancer". However, after spending $62 million, the project did not meet its goals and it has been stopped.

On February 8, 2013, IBM announced that oncologists at the Maine Center for Cancer Medicine and Westmed Medical Group in New York have started to test the Watson supercomputer system in an effort to recommend treatment for lung cancer.

On July 29, 2016, IBM and Manipal Hospitals (a leading hospital chain in India) announced the launch of IBM Watson for Oncology, for cancer patients. This product provides information and insights to physicians and cancer patients to help them identify personalized, evidence-based cancer care options. Manipal Hospitals is the second hospital in the world to adopt this technology and first in the world to offer it to patients online as an expert second opinion through their website. Manipal discontinued this contract in December 2018.

On January 7, 2017, IBM and Fukoku Mutual Life Insurance entered into a contract for IBM to deliver analysis to compensation payouts via its IBM Watson Explorer AI, this resulted in the loss of 34 jobs and the company said it would speed up compensation payout analysis via analysing claims and medical record and increase productivity by 30%. The company also said it would save ¥140m in running costs.

Several startups in the healthcare space have been effectively using seven business model archetypes to take solutions based on IBM Watson to the marketplace. These archetypes depends on the value generate for the target user (e.g. patient focus vs. healthcare provider and payer focus) and value capturing mechanisms (e.g. providing information or connecting stakeholders).

IBM Watson Group

On January 9, 2014, IBM announced it was creating a business unit around Watson, led by senior vice president Michael Rhodin. IBM Watson Group will have headquarters in New York's Silicon Alley and will employ 2,000 people. IBM has invested $1 billion to get the division going. Watson Group will develop three new cloud-delivered services: Watson Discovery Advisor, Watson Engagement Advisor, and Watson Explorer. Watson Discovery Advisor will focus on research and development projects in pharmaceutical industry, publishing, and biotechnology, Watson Engagement Advisor will focus on self-service applications using insights on the basis of natural language questions posed by business users, and Watson Explorer will focus on helping enterprise users uncover and share data-driven insights based on federated search more easily. The company is also launching a $100 million venture fund to spur application development for "cognitive" applications. According to IBM, the cloud-delivered enterprise-ready Watson has seen its speed increase 24 times over—a 2,300 percent improvement in performance and its physical size shrank by 90 percent—from the size of a master bedroom to three stacked pizza boxes. IBM CEO Virginia Rometty said she wants Watson to generate $10 billion in annual revenue within ten years. In 2017, IBM and MIT established a new joint research venture in artificial intelligence. IBM invested $240 million to create the MIT–IBM Watson AI Lab in partnership with MIT, which brings together researchers in academia and industry to advance AI research, with projects ranging from computer vision and NLP to devising new ways to ensure that AI systems are fair, reliable and secure. In March 2018, IBM's CEO Ginni Rometty proposed "Watson's Law," the "use of and application of business, smart cities, consumer applications and life in general."

Chef Watson

Chef Watson is Bon Appétit magazine's and IBM’s artificial-intelligence cooking web app. IBM’s collaboration with the Institute of Culinary Education to pair the expertise of chefs with cognitive computing, produced Institute of Culinary Education; IBM (2015). Cognitive cooking with Chef Watson : recipes for innovation from IBM & the Institute of Culinary Education. Naperville, Illinois. ISBN 978-1-4926-2571-1.

Chatbot

Watson is being used via IBM partner program as a chatbot to provide the conversation for children's toys.

Building codes

In 2015, the engineering firm ENGEO created an online service via the IBM partner program named GoFetchCode. GoFetchCode applies Watson's natural language processing and question-answering capabilities to the International Code Council's model building codes.

Teaching assistant

IBM Watson is being used for several projects relating to education, and has entered partnerships with Pearson Education, Blackboard, Sesame Workshop and Apple.

In its partnership with Pearson, Watson is being made available inside electronic text books to provide natural language, one-on-one tutoring to students on the reading material.

As an individual using the free Watson APIs available to the public, Ashok Goel, a professor at Georgia Tech, used Watson to create a virtual teaching assistant to assist students in his class. Initially, Goel did not reveal the nature of "Jill", which was created with the help of a few students and IBM. Jill answered questions where it had a 97% certainty of an accurate answer, with the remainder being answered by human assistants.

The research group of Sabri Pllana developed an assistant for learning parallel programming using the IBM Watson. A survey with a number of novice parallel programmers at the Linnaeus University indicated that such assistant will be welcome by students that learn parallel programming.

Weather forecasting

In August 2016, IBM announced it would be using Watson for weather forecasting. Specifically, the company announced they would use Watson to analyze data from over 200,000 Weather Underground personal weather stations, and data from other sources, as a part of project Deep Thunder.

Fashion

IBM Watson together with Marchesa designed a dress that changed the colour of the fabric depending on the mood of the audience. The dress lit up in different colours based on the sentiment of Tweets about the dress. Tweets were passed through a Watson tone analyzer and then sent back to a small computer inside the waist of the dress.

Tax preparation

On February 5–6, 2017, tax preparation company H&R Block began nationwide use of a Watson-based program.

Advertising

In September 2017, IBM announced that with its acquisition of The Weather Company's advertising sales division, and a partnership with advertising neural network Cognitiv, Watson will provide AI-powered advertising solutions.