The Code Breaking Crisis

And the Computer

What is a computer? This question probably seems like a silly question to most people in our time. A few decades ago most people's idea of what a computer was and what jobs it performed would have been very different from our idea of what a computer is today. If you have a dictionary that is over 20 years old the word "computer" probably will not be listed at all. The word "computing" will be listed but with a very different definition than we know today. Computing entailed different functions than computing does today. Computing until recently was synonymous with calculating in definition and in normal usage. Computing involved only the manipulation of numbers. The computers we now know are very new innovations. Why did it take so long to develop the modern computer? One reason was that the technology did not exist to produce the components necessary to build a computer until recently. Probably the main reason the development of the computer was hindered was that the high cost of research and development to bring us into the age of modern computing was beyond the limitations of private industry or individuals. Ideas for computer development existed but the funds to proceed did not. The impetus that triggered the government to step in and develop computers at any cost was the demands of modern warfare. I will attempt to describe the events and developments in modern warfare that proceeded the computer. The first computers were built under extreme pressure and they were built practically overnight if we use the span of a century as a reference. The reasons behind the massive effort that brought the world into the computer age required one civil war, two world wars and took almost a hundred years to culminate.

By the beginning of the American Civil War advances in weaponry and communications had already began to outstrip the technology that was needed to control and make full advantage of their usefulness. The developments of new and better propellants for use with small arms and especially artillery and naval guns necessitated the compilation of very involved tables of trajectory that sorely taxed the abilities of mathematicians. The manufacture of barrels and breech assemblies relied as much on the skill of the scientist as it had on the metal craftsman a few short years before. The invention of the telegraph and it’s introduction as a tool of warfare during the American Civil War seems to have taxed the mathematicians of the day as much as the innovations in weaponry.

In the book "The Codebreakers: The Story of Secret Writing" by David Kahn the author tells of the incredible volume of telegraph communications that occurred during the Civil War in a chapter titled "Crisis of the Union". Most of this communication was in codes of various types. The Union’s ability to decode Confederate messages while protecting their own transmissions greatly aided their efforts. Some vital messages were intercepted and decoded to late to avert disaster as was the case of the transmissions of John Wilkes Booth in reference to the assassination of President Lincoln. His message that he had killed the President was later found however. Kahn states that over six and one half million messages were transmitted by telegraph during the Civil War by Union forces alone. Most were in cipher or other codes. None of the union codes were ever broken by the Confederacy. All the codes of the Civil War were transmitted without the aid of any cypher devices. Codes most frequently used were word transpositions. The telegraph stimulated the invention of many new cyphers and of course new methods of cryptanalysis in attempts to break the codes. The first mechanical cipher device was patented in 1865, late to be of much use during the Civil War. This enormous amount of communication traffic would seem minuscule in comparison by the turn of the Century.

Ironically most of the early advances and use of codes and cyphers in pre-20th Century use were for business and not warfare. Pricing codes on retail merchandise and communication between businesses were just a few of the ways that the process of encoding information proved useful. Messages of length could be shortened with the use of codes and save businesses money on their communications expenses as well as maintaining confidentiality.

The telegraph was challenged by the invention of the radio transmitter/receiver and later by the teletype but would continue to play a crucial role in warfare whenever secure landlines could be maintained. Expertise in mathematics became an increasingly important element in waging a successful campaign. Understanding the science of modern communication became as important as understanding the complexity of modern ordinance. Commanders were finding that skill in communications was just as important as skill in weaponry. The skill of the mathematicians were essential in encoding and decoding the increasing volume of traffic generated during warfare.

World War I was the first protracted modern war to be fought. Many new inventions were given their first large scale field trials within the constraints of the First World War. The submarine, the airplane, the machine gun, gas warfare, long range artillery and the radio transmitter. While weapons had to wait to be tested in actual warfare codebreakers had been testing the codes of their potential adversaries with the use of stolen diplomatic transmissions. France held the German diplomatic codes by the eve of the First World War. France employed ten full time military officers as its cryptographers at the beginning of the war overseen by Captain Francois Cartier who was to become the head of the French cryptographic bureau during World War I.

Soon after its invention in 1895 the military had brought the radio into its arsenal. Radio messages have several advantages over telegraph or hand carried messages. Radio is instantaneous. Distance, terrain, enemy forces, the rapid movement of a mechanized army, navy or even air force are within the limits and capabilities of the radio. Miles and miles of wire were no longer required. An enemy behind the lines couldn’t cut the lines of communication. The list of advantages is long and for the most part self-explanatory. The list of disadvantages is fairly short but critical in the analysis of warfare in the 20th Century.

An enemy no longer had to gain access to a telegraph line behind the lines in order to intercept communications. Messages could be used to the advantage of anyone who could intercept and decode them. Many communications by radio were intercepted without the knowledge of the sender. It is bad enough to have your message intercepted but much worse if you don’t know it has been intercepted. Critical to the value of intelligence is the use of a cover-up employed to insure that the enemy doesn’t suspect that the communication has been compromised. Transmissions by radio although timely and able to transcend great distances are very vulnerable. To capture a message carried by a messenger or sent by telegraph a physical proximity to the enemy is required. Eavesdropping on radio communications can be done from the safety and comfort of a radio listening post at great distance from the hostilities.

Coding a message can be done in many ways. Simple transposition is one of the simplest. In transposition we simply change the letter orders of a message by jumbling. Another method of secreting a message is by substitution. In word substitution letters are normally replaced by other letters, numbers or combinations of letters or combinations of numbers. Codes use meaningful elements such as words and syllables while cyphers (or ciphers) concentrate on individual letters and numbers. In cypher words are normally split into disconnected letters. In cypher, substitution is used more widely than transposition. There is some fine hair splitting when we use the terms "code" and cypher" as both are interchanged in all but the most technical writing. By the beginning of World War I most messages were being transmitted in very complex cipher.

The codebreakers were of little use without intercepted messages to decipher. Although radio transmissions could travel beyond the horizon they were stilled called "line of sight" transmissions. A more recent term would be "directional". In order to intercept messages listening posts had to be on "line of sight’ of the radio broadcast. The first step in tapping the enemies radio communications was to employ radio signal detection and directional devices. Radio transmission detection devices were very crude at the beginning of the 1st World War consisting of antennas that rotated slowly in attempts at receiving a signal. Detection units employed hand held devices rotated by hand as well as motorized, vehicle mounted antennas. When a signal was detected it enabled the direction of the signal to be determined. if the signal could be detected from another location as well the exact ground location of the transmitter could be plotted. Jamming of enemy radio transmissions once the transmitter was located was an option as well as the interception and deciphering of enemy traffic. Messages were sent by means of Morse Code in an enciphered form. The personnel on the listening posts were not required to be cryptanalists but were required to be able to record messages in their initial form of Morse Code. Experienced listening post operators were able to recognize individual operators that were transmitting the messages in some cases providing context to messages. Telegraph operators all have distinctive keystrokes, a characteristic that would prove to be a valuable intelligence aid.

After a message has been intercepted and it’s Morse Code text initially recorded the cryptanalyst can go to work. Early pioneers of cryptanalysis such as Hassard, Holden, and Grosvenor had already broken ground by the year 1878 in deciphering double transpositions. The ease of French codebreakers in breaking German codes at the beginning of World War I began an almost mad and all consuming race to develop incredibly difficult ciphers that would proceed at an alarming pace until the end of World War II.

As the codes and ciphers became increasingly more difficult the amount of material and manpower allocated to codebreaking multiplied almost exponentially. Armies with modern transportation outstripped the ability of telegraph as front lines were advanced while ships at sea were for the first time able to communicate great distances by radio. All services transmitting did so with code. To avoid confusion and lessen the effects of compromise each branch of the services employed their own codes. Suddenly a few cryptanalysts deployed at the beginning of the war could not make a dent in the amount of material being transmitted. In most cases the Signal Corps of nations being expert in the use of telegraph and radio were given the responsibility of breaking the codes of the messages they intercepted. What had been small branches of the services became priority specialties employing many men of various degree of skill depending on their assigned duties.

An intercepted enciphered note to the German minister at Mexico City known as the Zimmerman telegraph was carried to Congress by President Wilson to convince them of the hostile intentions of Germany toward the United States. The United States entered World War I partly due to this intercepted telegram that include the American Southwest within contingency plans if war should break out between the United States and Germany. It is said that no other single cryptanalysis has ever had such enormous consequences.

The telegraph played an important role for the French Army in W.W.I. Germany invaded France and outstripped it’s own telegraph system in it’s push into France. The French being driven back and fighting the war from their own territory could still use their telegraph system that remained almost completely intact behind their lines. German communications were broadcast to units in the field almost entirely by radio in the forward positions. The amount of easily interceptable German traffic allowed the Allies more opportunity to break the code.

America sent it’s A.E.F. force to Europe with its signal branch and it’s original codes and ciphers. The Americans were fairly successful in maintaining the integrity of their codes with a tactic of frequent changes and tight security of materials used to develop the codes.

One American named Herbert Yardley who had worked in codebreaking during World War I continued after the war in secret intelligence activities known as the "Black Chamber" sponsored by United States Army Military Intelligence. Their primary goal was to gather information from radio and telegraph intercepts of Japanese communiques, both military and governmental. Relations between the U.S. and Japan had already began to deteriorate and some within the government of the United States were already preparing for the possibility of war. This clandestine operation was not in keeping with international law or even the laws of the United States at the time and so was disbanded within a short time of discovery of the unit's existence by civilian politicians. In the interim valuable information was gathered and experience gained that seriously weakened the integrity of Japanese transmissions by the beginning of World War II.

Ciphering devices has been in use since the 4th century B.C. and progressively became more complicated. As I stated earlier the first telegraphic cypher device was invented in 1865. The first electro-mechanical device was invented by three men at almost the same time. American Edward Hugh’s machine was invented in 1918 and continued to be used in this country through W.W.II. Arvid Damm of Sweden built a machine in 1919 selling it later to the Swiss Boris Caesar. Hugo Kock of the Netherlands invented another version of the same machine in 1919 and later sold his machine to Arthur Scherbius. It was Scherbius’ machine that upon improvement became the famous German Enigma Machine. Arthur Scherbius had built a machine for business transmission that enabled messages to be sent in confidence without complicated codebooks. The first machines were a series of drums and stepped rings labeled with numbers and letters turned by hand crank. Although very efficient they were too slow for the more complicated codes and especially after additional rotors were added as the race to create more sophisticated cypher devices quickened. The goal of each nation was to invent an unbreakable code and the system to transmit it. As time went on the rotors were made to be removable so they could be re-positioned in different orders. Positions of the rotors was critical as correct positioning of the rotors also acted as a switch for an electrical current to pass and advance the machine to the next level of settings. The Enigma machine settings had two overall dominant positions that of encipherment and decypherment. As if this weren’t enough combinations and possibles to stump the cryptanalysts; the Germans added another complication they named the "Steckerboard". The Steckerboard was a plug board with jacks similar to a telephone switchboard. By cross-connecting the rotors and any combinations of two letters on the keyboard they could increase the complexity and difficulty required to untangle any combination of 12 letters they chose dramatically. The number of possible encipherments was raised to 10 quadrillion. It was estimated that if 1000 operators with the same identical captured machines tested four keys a minute for 24 hours a day it would take them 900 million years to try them all. Germany was convinced its code machines transmissions were unbreakable. The Polish Cipher Bureau cracked large portions of the codes between the wars before electro-mechanical rotor devices came into use. The information the Poles had collected was handed over to Britain at the beginning of World War II. The name of a article written by Bill Momsen on line that has more detail is "Codebreakers and Secret Weapons in World War II, Chap. 2: 1939-1941" and the address is <http://members.aol.com/nbrass2enigma.htm>. The Germans were to find later that the machines would only be as good as there operators.

Other Axis members shared the use of the German Enigma Machine. Among this group were the Japanese. Having been given an early German commercial version the Japanese modified it and put it to use with their diplomatic service. America agreed in 1940 to exchange a reproduction of this Japanese machine codenamed "Purple" with Britain in exchange for information on the newer German version of Enigma. In David Kahn’s book titled "Seizing The Enigma:The Race to Break the German U-boat Codes, 1939-1943" I found that while British efforts had been focused almost entirely on intercepting German transmissions and breaking those codes American had focused all but 20 percent of it’s interception efforts and 3 percent of it’s codebreaking efforts on the Japanese transmissions. Communications between Germany and Japan although considered high level and accordingly in more difficult, cypher proved a valuable asset to the Allies. The Japanese Naval Attache’in Berlin code was even broken revealing details about German rocket factories according to "Codebreakers:The Inside Story of Bletchley Park"edited by F.H. Hinsley and Alan Stripp.

Admiral Yamamoto planned the attack on Pearl Harbor and was instrumental in the Battle of Midway. The Pearl Harbor attack had proved successful for Japan in a large part because they had observed radio silence while closing for the attack. Time and again when German and Japanese units observed radio silence most operations were successful. When units were liberal in the use of radio transmissions the results were disastrous for them. Such is the case in the events that led to the Admiral’s death.

Admiral Yamamoto’s forces were defeated at Midway by brave U.S. pilots and with the intelligence provide by a I.B.M. tabulator located in the basement of the 14th Naval District’s Administration Building located at Pearl Harbor. A tabulator is a typewriter device that puts figures into columns. The tabulator speeded up the work of the cryptanalysts. The machine and codebreakers working together were able to give the United States vital intelligence prior to the battle. When the Japanese code appeared to be broken a baited message was sent in the open. Japanese transmissions by radio in response to the interception of the open radio message sent by the U.S. were monitored to confirm that the code was actually broken. The validity of vital captured information was confirmed in this way. These events allowed the U.S. to maneuver its forces wisely during the attack at Midway. Because of the valuable intelligence it had acquired the United States was victorious at the Battle of Midway. Many historians believe Midway was the turning point of the war in the Pacific.

Admiral Yamamoto soon went to his death because of the same I.B.M. tabulator that had beaten him at Midway. Units preparing for the inspection visit of their Commander and Chief exchanged his schedule by radio. These messages were intercepted and decoded. American P-38 aircraft were sent to intercept Admiral Yamamoto’s plane enroute over Bouganvlle and he was shot down. Many historians believe that Yamamoto’s death was the second major blow to Japan’s ability of waging war in the Pacific. You can view a photo of Admiral Yamamoto by going to this address,

<http://www.skypoint.com/members/jbp/officers.htm>.

Just as critical to the outcome of the Second World War, the war in the Atlantic raged on. The Battle of the Atlantic was fought to protect shipping that was being destroyed by U-boats faster than it could be built. You can view a drawing and photo of a U-boat by going to this site; <http;//www.cshost.com/~grey-wolf/uboat/utyp-1a.html>. The allies of the United States in it’s war against the Axis forces needed war materials and foodstuffs produced in the U.S. The only way to transport the material in the huge tonnage required was by sea. German "wolf packs" (submarine groups) preyed on ships coming to and going from the United States. Merchant ships were lightly armed whenever possible and organized into convoys that were escorted by naval vessels. In spite of the precautions that were taken these ships were easy targets for the German U-boats. Submarine hunting aircraft were employed by the Allies to spot U-boats as they surfaced to recharge batteries and refuel or when they were surfaced to attack un-escorted shipping with their deck guns. Sometimes submarines might stay submerged for weeks at a time. A partially submerged submarine is very difficult to spot but if it sends any radio messages they can be intercepted. As efficient killers as the U-boats were one of there few weaknesses was the radio. Listening posts could pinpoint their positions alerting convoys to skirt the packs and anti-submarine units could be deployed in pursuit. Some experts maintain that the Battle for the Atlantic was won only because the Allies could intercept and read the naval cyphers. Early in World War II no amount of intelligence could have stopped the German advances on land, sea and air due to the vast numerical advantage, superior equipment, and training. Later in the war as German attrition and the destruction to Germany’s infrastructure became more deliberate the Allied intelligence began to have more of an effect in helping to win the war.

British codebreaking had been conducted by Naval Intelligence in the famous "Room 40" during W.W.I and transferred to the Foreign Office in 1920. Research on the codes and cyphers of all the major powers including Japan and Germany were carried out by the Foreign Office. Efforts were unsuccessful in the case of the German Enigma Codes until 1939 when an Enigma replica and codebooks were turned over to Britain by three Polish mathematicians who had escaped to England after the Nazi invasion of their country. In August 1939 codebreaking operations were moved to Bletchley Park, an old English estate, about 40 miles from London. Bletchley Park was chosen for two main reasons. As the prospects for war between Britain and England grew more likely it was decided that a location out of the probable zone of Axis saturation bombing would be necessary. In addition the site must have room for the expansion of facilities if war did come. An excellent photo of the old home (mansion) at Bletchley Park can be accessed by going to the Bletchley Park Trust’s homepage and navigating to the history link. The address is <http://www.bletchleypark.org.uk>.

What really made Bletchley Park different than other facilities wasn’t it’s location and it wasn’t the mansion on the grounds. What made Bletchley Park unique was the assembly at that place of some of the most brilliant minds of that or any other time. Of the 8000 people that worked at the facilities processing 10,000 messages a day at the peak of the war several individuals stood out from the rest because of their genius. One of these was Tommy Flowers a Post Office engineer. In the 1930s Tommy Flowers investigated electronic telephone transmitting devices. Flowers was asked in 1942 to mechanize whatever portions of the decyphering he could and eliminate some of the work being done by hand. The famous mathematician Alan Turing thought that improvements could be made to an electromechanical device which the Poles had begun experimenting with called the "Bombe" and approached Flowers for technical advice. The Bombe was designed to test possible solutions to cyphers generated by the German Enigma machines. Work on the Bombe was only the beginning of Flowers work with the codebreakers. The nickname for intelligence gained from the Enigma transmissions was called "Ultra". In 1941 valuable Ultra intelligence told of the German plans for attack in Greece that allowed the out numbered British onsite to evacuate before they were surrounded. Ultra intelligence also helped defeat Rommel in North Africa. Ultra intelligence led to the discovery of a German radio guidance beam that could lead bombers to their targets in England. Once discovered, the beam could be "jammed". The best known successes of Ultra intelligence gained with the assistance of the Bombe machines is the part they played in the war against the Wolf Packs of the Atlantic. Alan Turing had 6 Bombes at Bletchley Park and surrounding villages. Bombes were also in place at Stanmore and Eastcote England.

While the Bombe was effective in decyphering Morse Code based Enigma communications it was to slow to be of real use in decyphering new high level Non-Morse based transmissions sent by a teletype device called the Lorenz SZ which was faster and much harder to break. The typewriter like Enigma device was capable of 159 trillion possible combinations. The Lorenze device was capable of producing 26 that many combinations. to process one message therefore would have required 26 times more Bombes. The Bombe was outgunned.

Human beings have the capability and capacity to figure out anything if given enough time. That has been proven many times throughout history. During World War II a quantity always in short supply was time. In World War II no location had been safe from U-boats. HMS Royal Oak, a battleship was sunk by a German U-boat at Scapa Flow the homeport of the British Fleet and an aircraft carrier the HMS Courageous was sunk off the coast of Ireland early in the war. In 1940 Hitler declared unrestricted warfare in English waters. a U-boat commander named Otto Kretschmer of U-99 sank 44 ships during the war. In one three-month period of 1940 over one million tons of shipping was destroyed by U-boats. Just when the British began to get the upper hand on the U-boat problem with the aid of captured and decyphered communications the Germans would make a major change in the codemachines. The Allies managed to capture parts of Enigma Machines several times during the war and 1 totally in tact machine was captured from the German U-boat 110.

The introduction of the Lorenz system and its complexity minimized the successes the British had enjoyed against Enigma because of the Bombe.

Intelligence gained from the Lorenz SZ system with it’s 32 letter alphabet was codenamed "Fish" after the roughly pronounced name of a manufacturing company printed on a component of a German machine which had been captured. Everyone at Bletchley Park was to concentrate on "Fish". Fish data was figured by hand until mathematician Bill Tutte gained some insight into the Lorenz system from an intercepted series of transmissions. At that point a two-part project was begun with the linguist Ralph Tester heading one department while mathematician Max Newman headed another overseeing Tommy Flowers in his attempts at automation.

The teams developed a machine with 24 switches (called valves) named the "Heath Robinson" used in combination with a switchboard-like device called the"Tunny". This machine processed messages run through it on narrow paper tape with punched holes that represented the electronic pulses sent by the Lorenz machines.

The engineer Flowers realized that an electronic signal could be generated that duplicated the original message that had been intercepted. Optical sensors could then be used to read the signals. This breakthrough was significant but the need for paper tape strips for data entry meant that a machine could only work as fast as tape could be fed through it.

Another machine was developed at Dollis Hill that had 1,500 valves switches). By the end of 1943 the first of 11 machines had been installed at Bletchley Park. The machine was the size of a small room and weighed about a ton and was aptly called "Colossus". While still very primitive it could accomplish tasks that had taken weeks of work previously in a few hours.

Allen W.M. Coombs describes Colossus in his article "The Making of Colossus" as being a very innovative machine. The vacuum tube switches were thought to be too delicate by most electrical engineers excepting a few engineers like Flowers and Coombs. While the Mark I Colossus was getting underway in the month of December 1943 Coombs was already beginning separate work on Colossus Mark II. The Colossus Mark I which could function well at well above 2000 characters a second was needed online immediately and could not be refined or improved further without delaying the immediate mission. Meanwhile voltage fluctuation problems were solved by installing transformers within the system and special heaters were designed to preheat the vacuum tube switches to increase their life and performance. A major problem in the building of both the Mark I and Mark II was that many components that were needed were in short supply or hadn’t been invented at the time such as the special resistors needed. Ideas such as storing information as small electrical charges within capacitors and using remote keyboards to direct the machines were ahead of their time. The Mark I and II have the distinction of being the first computers ever built that could be controlled from an internally stored program. The program could not store a program effectively once switched off and so had to be kept on line continuously or be reprogrammed if shut down. Experimentation into electronics fabrication ultimately produced faster models such as the Colossus Mark II in time to be of great value to the war effort.

Sir Harry Hinsley, Bletchley cryptanalyst and historian, states that Colossus was instrumental in the planning of the D-day invasion. German troop movements and strengths were considered high level communications and were transmitted on the Lorenz system that had been impossible to crack without the Colossus type computer. Not only were the high level communications hard to crack but German Army communications in the form of landlines had been in place for quite some time in the occupied countries by the time of the invasion. Enigma codes for the Army that could have been solved on the Bombe systems weren’t being transmitted by radio with any frequency. This meant that almost all Allied intelligence from transmissions came about through the breaking of "unbreakable" Lorenz transmission. The Mark II with 2400 valves (switches) was in operation by the invasion during June of 1944 to quickly decode any transmissions that could be intercepted. Many feel that intelligence gained by the Bombe and Colossus projects saved countless Allied lives and may have shortened the war by almost two years.

It is said that the binary logic of the Colossus series of computers could be handled by the tiniest microchip of today. In it’s day however it was considered very powerful and even dangerous in the wrong hands. At the end of the war the eleven Colossus type machines in use were destroyed by Winston Churchill’s orders. In the postwar years the four technological developments that would define the strategic environment were the ballistic missile, the atomic bomb, radio electronics and the electronic computer.

What had begun as an effort of 100 people in 1939 had grown into an intelligence effort employing over 7000 people in 1944. Many of the great minds of the time were congregated together in the "huts" of Bletchley Park.

In the race to break the codes of the German High Command and the U-boats, science and mathematics had taken the first steps toward the research and creation of the digital computer.

The story of the Bletchley Park is not over. Included in the Bletchley Park Trust website is the following statement:

"In May of 1991, the Bletchley Archaeological and History Society

formed a small committee to bring together as many former codebreakers as could be traced, for a farewell ‘thank you’ before the site was destroyed. On 21 October 1991, the farewell party was held on the grounds. Over 400 codebreakers attended. As a result of the stories they told, it was decided to attempt the restoration of the site for posterity. The Bletchley Park Trust grew from this small committee."

The Bletchley Park restoration isn’t just about buildings it is about the forerunners of today's computers. In a 6 page article accessible through the Park homepage Tony Sales tells the story of Colossus and the rebuild that took place under the guidance of Trust members many of which are former designers and operators. Access this site directly at the following web address. <http://www.cranfield.ac.uk./ccc/park/colossus.htm>.