APPENDIX F: AMERICAN INTELLIGENCE IN THE PACIFIC WAR
Because of the astonishing success of Captain Jo Rochefort and his team’s decoding of Japan’s main navy code before the Battle of Midway, the most important American victory of the Pacific War, it is easy to overlook the pervasive role of intelligence throughout the conflict. The purpose of this appendix is to briefly summarize the constant importance of ULTRA, the compendium of intelligence available daily to America’s senior commanders. Indeed it would be appropriate to describe ULTRA as the oracle, which was consulted prior to every significant operation in the Pacific War. Indeed in many cases ULTRA intelligence initiated tactical and strategic operations.
Although ULTRA, is sometimes referred to as if the breaking of Japan’s main navy code, JN-25, was its only manifestation, it was in fact a code name for many sources of information; it was in effect a daily compendium of intelligence. ULTRA, was the name given to ‘ultra’ special intelligence including information from captured documents, coastal watchers, analysis of Japanese the weight of Japanese call traffic in different locations, and interviews from POWs (though this was relatively limited). Traffic analysis would include studies of procedures, call signs, and plain un-coded messages. ULTRA also included the MAGIC decrypts from the encryption machine known to the Allies as PURPLE – a near equivalent of the German Enigma machine. Japanese civilian radio stations were also monitored. Given the vast range of the Japanese Empire, which reached thousands of miles into the Pacific, and the absence of undersea cables and often overland ones as well, it was necessary for Tokyo to broadcast at high frequency to reach Imperial outposts. Allied listening posts in the Aleutians, Hawaii, Ceylon, the west coast of America, and Australia were able to pick up huge volumes of traffic. Geographic location of volume as well as direction could give vital clues. Changing patterns in this traffic could give evidence of strategic and indeed tactical movements.
Just as ULTRA was a complex compendium of intelligence information, so too was the analysis of Japanese codes. JN-25 was the name given to the main naval code, which was used for 70 per cent of all traffic. However the quite separate Flag Officer’s Code was reserved for senior commanders and despite the best efforts of the Combat Intelligence Centre (Station HYPO) run by Joe Rochefort on Hawaii, remained largely though not entirely impervious to being broken during the course of the war. JN-25 itself was far from being a ubiquitous single code such as that used by the Germans with their Enigma machine enciphering nearly all the communications of the Kriegsmarine, Wehrmacht, SS, police, the diplomatic service and the Luftwaffe. By comparison Japan, in four years of war, had used more than twenty-five different codes or variations for use by different services or operatives. The Japanese language itself was another barrier. Though Japanese people, always incredulous that foreigners were capable of learning their language, perhaps overestimated the usefulness of this barrier. In Britain, America and Australia, fluent Japanese speakers emerged.
Often overlooked in the history of ULTRA and the breaking of the Japanese Navy Codes is the role of Government Code and Cyper School (GC&CS) at Bletchley Park in England. This organization, made recently famous by the cinematic story of Alan Turing and the breaking of the German Enigma codes, The Imitation Game , broke Japanese diplomatic traffic in the 1920 and subsequently worked on Japanese Naval codes. The section was operated by Englishmen William ‘Nobby’ Clarke, Harry Shaw and Ernest Hobart-Hampden and were later joined by Australian crypto-analysts John Tiltman and Hugh Foss. They worked with the US Navy Code and Signals Section that was know as OP-20-G in Washington D.C. and FRUMEL (Fleet Radio Unit, Melbourne). An Asian based unit of GC&CS also operated first in Hong Kong, then Singapore and finally in Colombo (Ceylon) and Kilindini (East Africa). GC&CS worked with Operation HYPO and other intelligence groups, notably one based in Brisbane, on the cracking and analysis of JN-25.
Codebreaking is a group activity and attribution of key breakthroughs to individuals is often shortcut to a more complex truth. Just as Turing has suddenly gone from obscurity to the embodiment of all of the code breaking success of Bletchley Park, a gross simplification, so the breaking of Japanese naval codes has tended to be attributed solely to Jo Rochefort in Hawaii. Suffice it say that the JN-25 code was cracked by Rochefort and his team (code named HYPO) together with the other American, British and Australian code breakers by November 1940. However they had less than a month to give up its secrets. JN-25A was replaced by JN-25B on 1 December 1940. Work began immediately to decrypt the new version but it was only partially broken by the time of the attack on Pearl Harbor on 7 December 1941. From mid-1941 the intelligence that had been gathered from decrypts and other sources came to be called ULTRA in its collated form. Eventually ULTRA grew into a monstrous intelligence resource, which could even report on individual Japanese promotions and requests for mechanical parts for ships. By the end of the war, the US had logged some 290,908 decrypts of Japanese signals.
It should be noted that because of the radio silence imposed on the Japanese Combined Fleet before the attack on Pearl Harbor, there was no traffic to decrypt from the carrier force that destroyed the US Navy at anchor in Hawaii. Nevertheless traffic in other areas gave up clues, which Rochefort believed in hindsight would have given up the secret attack if US intelligence operations had not been so badly riven by poor communication and management. He opined that pre-war political infighting was largely to blame.
Despite failures in this instance, the importance of ULTRA led to extraordinary measures to protect it. Unless another source, such as a reconnaissance mission could have provided intelligence, commanders had to refrain from action that could be perceived by the Japanese as a sign that their codes had been broken. Therefore ULTRA was used sparingly and only when urgently required. The use of ULTRA in the interception of Japanese submarines was particularly fraught with the risk of giving away the US code breaking ability as there were few, if any, other means by which Japanese movements could be discovered. Furthermore even though messages sent by US armed forces the word ULTRA as little as possible, the word needed to be used in different ways in each sentence, so its meaning could not be deduced from any recurring pattern. Access to ULTRA was off limits to all but a chosen few. Famously Rear Admiral Arleigh Burke, Admiral Mitscher’s Chief of Staff, raised a rumpus when he found that his boss was secretly conversing with a lowly intelligence officer, Charles Sims, behind his back. Burke protested to Nimitz that he should be let in on their cabals and despite Sims’s protests, permission was given for him to join the select club who had access to ULTRA. It was later revealed that whereas MacArthur in the Philippines had access to this highest-level intelligence, which showed that Japan was mobilizing for war, Admiral Kimmel did not. It made it doubly ironic therefore that Kimmel took the fall for the catastrophe at Pearl Harbor while MacArthur was lionized in spite of his lack of preparedness in the Philippines debacle.
By luck JN-25B was decrypted far quicker than its predecessor. Japanese operators made a mistake by sending an enciphered message in plain text simultaneously. The result was that within a month of Pearl Harbor, the main Japanese naval code had again been broken. This did not mean that 100 per cent of all messages became decipherable; often intelligence breakthroughs came down to piecing together information from as little as 10 – 20 percent of any message – and only 60 percent of all messages were captured at all. The assembly of information into ULTRA therefore often resembled the piecing together of a jigsaw despite many pieces being missing.
In spite of the large quantum of guesswork involved, the compilation and use of ULTRA soon started to produce results. Admiral Nimitz’s intelligence adviser, Commander Edwin Layton and Captain Rochefort gathered from intense radio traffic activity early in 1942 that Rabaul was the new target of the Japanese Combined Fleet. With the Japanese thus occupied in the eastern islands of New Guinea, Nimitz could therefore authorize Admirals Halsey with USS Enterprise and Fletcher with USS Yorktown to attack the Marshall and Gilbert Islands some 2,000 miles distant. These may have been small prizes but they raised the morale not only of the fleet but also all Americans.
The first major contribution of ULTRA came with the revelation that the Japanese planned to launch an amphibious attack on Port Moresby, the last Australian stronghold on New Guinea, at its southeastern tip. The resulting Battle of the Coral Sea may have been a US naval defeat for a first ever action between carriers but it was a strategic victory in that the Japanese troop transports were turned backed. It was their first and last attempt to take Port Moresby by sea. As has been thoroughly noted in Chapter 12: Battle of the Coral Sea: The First Carrier ‘Shoot-Out’, the outcome of the battle with the heavy loss of aircrews also robbed Admiral Yamamoto of its two most modern carriers at the subsequent Battle of Midway. The outcome of that battle and the crucial role of ULTRA intelligence in helping to achieve it has been extensively described in Chapter 13 Battle of Midway: Nimitz’s Lucky Day and need not be repeated here. But it is worth again noting that it led to Rochefort being described as the MVP (most valuable player) in this turning point engagement.
The importance of ULTRA is perhaps best shown when it was not available, or at least not available with information from the decryption of JN-25 incorporated. This became evident after 28 May 1942 when Japanese intelligence moved from JN-25B to JN-25C. The fog of war suddenly became a pea-souper. It did not help that at the end of September 1942 Japan reorganized its entire communication system, which meant that many of the aids to US intelligence analysis such as call signs disappeared overnight. Frustratingly from the point of view of HYPO, Rochefort and his crew had to restart with a clean sheet. Throughout the Guadalcanal campaign US forces operated in the dark – and it showed. The almost catastrophic defeat at the Battle of Savo Island happened because the US-Australian fleet was taken completely by surprise. The loss of information from JN-25 did not curtail all of ULTRA usefulness but intelligence could not be provided with the same degree of accuracy and reliability as at the Battles of the Coral Sea and Midway.
By late autumn it was evident that Rochefort had recouped some decryption capability and on the 23 October ULTRA warned that a new Japanese offensive against Guadalcanal was very likely. In spite of this, the Battle of the Santa Cruz Island resulted in the loss of USS Hornet and a tactical defeat for the US Navy. The battle, which demonstrated the limits of intelligence in determining the outcome of naval engagements. That the precious advantage of advance knowledge could be thrown away was again shown at the last naval engagement of the Guadalcanal campaign, the Battle of Tassafaronga. Despite being surprised, the Japanese destroyer force led by Admiral Tanaka wrought havoc with Long Lance torpedoes on a much stronger cruiser squadron led by Admiral Wright. Subsequently, sign changes on 1 February 1942 meant that there was no advance intelligence of Japanese withdrawal from Guadalcanal on 7 February.
Thereafter ULTRA provided key intelligence and enabled the next major turning point of the war – the establishment of US air dominance over the Solomon Sea and the Gulf of Huon. ULTRA predicted with total accuracy that six troop transports were bound for Lae on the Huon Peninsula and expected it to arrive on 5 March. Reconnaissance planes looked out for and found the Japanese expedition and tracked it to its ultimate point of destruction the Battle of the Bismarck Sea.
Second only to the Battle of Midway as the most famous ULTRA contribution of the Pacific War is the assassination of Admiral Yamamoto. Captain Yasuji Watanabe, a close friend of Yamamoto, insisted that the details of his tour of inspection of the airfields of Bougainville be transmitted only in Navy code. By mistake they were sent by Flag Officer’s code as well. FRUPAC (Fleet Radio Unit Pacific) as HYPO was now known intercepted a fragment of a message on 14 April 1943: “On 18 April Commander in Chief Combined Fleet will ----- as follows: Ballalae Island ------ ---- ----.” More detail followed and with the usual jigsaw deductions Layton and Rochefort worked out the exact route and timing of Yamamoto last fatal flight. See Chapter 21: Yamamoto Assassinated and the Battle of New Georgia.
A lesser-known success involved the undersea war on Japan’s transport lifeline, which was transformed by an ULTRA breakthrough in January 1943. The cracking of the code for Japanese merchant ships (Maru) known as the Maru Code led to a dramatic rise in sinkings. In the 14 months to the end of January 1943 just 140 Japanese merchantmen were sunk; by contrast 55 were sunk in February 1943 alone and 72 in March. It is interesting to note that the improvement in US submarine performance began before the torpedo problems of Newport, Rhode Island had been solved, though clearly having torpedoes that worked massively increased the potency of the US submarine campaign. But Vice-Admiral Charles Lockwood, Commander-in-Chief Submarines Pacific, paid particular tribute to the contribution of the intelligence services. In spite of the fear of overusing intelligence from the decryption of the Maru Code, it is estimated that as much as 50 per cent of all submarine kills were guided by ULTRA. Indeed the relatively poor performance of the Japanese submarine force may in part have been due to the fact that they could not match the Allies’ decryption skills.
As the war progressed, ULTRA became a less important factor in the outcome of major engagements. At the Battle of the Philippine Sea, ULTRA placed Admiral Ozawa’s advancing Japanese fleet on a pinhead, though arguably by this stage America’s new fleet, which had been forged in the dockyards of the United States since the beginning of the Pacific War, was so technologically advanced that it no longer relied so heavily on intelligence. By the time of the Battle of Okinawa in March 1945, ULTRA was able to provide large swathes of information about planned kamikaze attacks but such were their overwhelming numbers that losses suffered by the US Navy were still the largest of any naval battle in their history.
Indeed, an ULTRA shortcoming played a part in a more direct failing related to the sinking of the USS Indianapolis. It took four days for naval authorities to pick up on the plentiful intelligence that the sinking of the Indianapolis had taken place. Shortly after it had planted its torpedo, Japanese submarine I-58 had signaled triumphantly that it had sunk a major capital ship – it was information that was passed to CINCPAC in Hawaii within sixteen hours of the event. Back in Hawaii it was dismissed by intelligence officers as yet more false reporting by Japanese submarines. The result was that most of the crew of USS Indianapolis died from thirst or shark attacks leaving just 316 survivors.
However it was a rare failure. Overall the ULTRA material provided by the intelligence services to be one of the defining advantages won by US forces during the Pacific War. As long as the United States maintained the will to defeat Japan, their victory was always likely but it seems quite possible that the key turning points of the Coral Sea, Midway, the Battle of the Bismarck Sea and the underwater war may not have swung in America’s favor without the advantage of ULTRA. Would America have been so enthusiastic about pursuing Japan’s unconditional surrender if their advance toward Japan had been delayed by several years? Although this is an unanswerable question, reflecting on it does underline the importance of ULTRA to the American war effort in the Pacific.