A number considerably above the average

To achieve a million once may be regarded as good fortune; to do so twice looks like carefulness, skill, enthusiasm, and dedication.

We’ve been working on the US Arctic logbooks since July 2012, and in that time we’ve rescued one million weather observations (actually 1,081,641 as of March 11th). That’s an awesome achievement – even more awesome than it seems at first, as a single weather observation is a complex object containing a lot of information. Our 1,081,641 observations comprise:

  • 960,221 air pressures,
  • 2,782,755 temperatures (950,351 dry-bulb, 765,270 wet-bulb, 597,556 barometer and 469,578 sea),
  • 1,020,223 present weather codes,
  • 1,049,692 wind directions,
  • 1,019,268 wind forces,
  • 864,586 cloud fractions, and
  • 846,859 cloud types.

To record all that, we’ve typed 21,511,795 characters – about 20 times the length of a typical novel.

However you look at it, that’s a tremendous achievement.

Code-breaking and oldWeather

Marcus Cocceius Nerva Caesar Augustus. 13th Caesar.

Marcus Cocceius Nerva Augustus. 13th Caesar.

Jr nyjnlf unir na rlr gb gur shgher, urer ng byqJrngure, naq jr’ir abgvprq bhe sevraqf ng gur nepuvirf cubgbtencuvat fbzr arjre ybtf – H.F. fuvcf sebz gur zvq-20gu praghel – nf jryy nf gur byqre barf jr’er hfrq gb. Gur nepuvirf ner abg cubgbtencuvat gurz sbe hf – gurl ner cneg bs nabgure cebwrpg – ohg gurl ner trarebhf crbcyr, naq gurl jvyy tvir hf n pbcl bs gur ybt obbx vzntrf vs jr jnag gurz. Qb jr jnag gurz?

Jryy bs pbhefr jr qb – bhe ybir sbe ybtobbxf arire snvyf, naq jr ner qrgrezvarq gung gur xabjyrqtr va gurz jvyy abg cnff njnl. Ohg jr pna’g qb rirelguvat evtug abj. Fb jr qb, hasbeghangryl, unir gb or frafvoyr, naq frg cevbevgvrf, naq lnqn lnqn lnqn.

Gur zbfg vzcbegnag guvat gb xabj nobhg nal cbffvoyr arj ybtf, vf jurgure nalbar unf ybbxrq ng gurz orsber: unir gurve jrngure bofreingvbaf nyernql orra genafpevorq? Gb svaq bhg, jr arrq gb ybbx va VPBNQF (gur vagreangvbany qngnonfr jurer jr fgber bhe jrngure erpbeqf) naq frr jung gurer vf sebz nal fuvc jr ner vagrerfgrq va – ohg gurer’f n pngpu.

Zbfg bs hf ersre gb n fuvc ol vgf anzr; bapr gur bjare unf qrpvqrq gb anzr n fuvc UZF Jbatnaryyn gura gur erfg bs hf fubhyq whfg sbyybj nybat. Bhe cerqrprffbef, ubjrire, jub qvtvgvfrq fbzr ybtobbx jrngure erpbeqf qrpnqrf ntb, qvq abg nterr. Creuncf gur anzrf jrer gbb ybat gb svg bagb gur Ubyyrevgu pneqf gurl hfrq sbe qngn fgbentr, ohg jungrire gurve ernfba gurl hfrq pbqr ahzoref nf fuvc vqragvsvref. Fb vs jr jnag gb svaq gur erpbeqf bs gur HFF Nexnafnf va VPBNQF, jr arrq gb xabj gung ure pbqr ahzore vf 01033 – naq, va trareny, jr qba’g.

Ohg gur byqJrngure grnz ner n pncnoyr ohapu, naq Xriva naq Znex ner sngubzvat guvf zlfgrel. Jvgu uryc sebz VPBNQF rkcregf ng APQP naq ABPF gurl ner qrpbqvat gur VPBNQF vqragvsvref fb jr pna yvax gur byq bofreingvbaf jvgu gur Nepuvir’f ybtobbxf naq cvpx bhg gubfr fgvyy haernq.

Certificated (2)

Data.Rescue.Award

Thanks again to the sponsors, organisers, and judges, of the 2013 International Data Rescue Award in the Geosciences, for adding to our project laurels.

Old New Zealand, HMS New Zealand, & new New Zealand

Atmospheric pressure along the route sailed by HMS New Zealand in 1919. The blue band shows the range of our estimates before oldWeather, the black points the new observations we provided, and the red band the revised analysis range incorporating our observations.

Atmospheric pressure along the route sailed by HMS New Zealand in 1919. The blue band shows the range of our estimates before oldWeather, the black points the new observations we provided, and the red band the revised analysis range incorporating our observations.

This week, atmospheric scientists are gathering in Queenstown, New Zealand, for the fifth general assembly of the SPARC program (Stratosphere-troposphere Processes And their Role in Climate). We’ve mentioned New Zealand before: both as a country who’s isolation means that its historical weather is poorly documented, and as a Battlecruiser in the original oldWeather fleet. In September 1919 the two met: the battlecruiser visited the country, giving us an opportunity to make a major improvement in reconstructing the climate of the region.

As we showed back in October, we’re now re-doing our analysis of global weather, so we can see exactly how much the observations we’ve recovered from HMS New Zealand have improved our knowledge of the climate of New Zealand (the country). The figure above (made for the SPARC meeting) shows our estimates of the weather in each region visited by HMS New Zealand during her circumnavigation in 1919: blue for before oldWeather, and red a new revision using our observations. The width of the band indicates uncertainty – narrower is better – and the improvement we’ve made is very large.

Documenting World War 1 at Sea

We chose our first batch of logbooks to cover the period of the First World War, as our climate records from this period were particularly poor. This gave us not only invaluable new climate information, but also a new look at a key historical period which is about to reach its centenary.

To use our historical results, we teamed up with Gordon and Naval-History.net. In Gordon’s words: ‘Our present world has been shaped by World War 1 – as much a maritime war as World War 2. Not just the Battle of Jutland or the Allies near-defeat by the U-boats, but Mediterranean, Belgian coast, South West & South Africa, East Africa, Persian Gulf, German raiders, Atlantic convoys, North Russia.’ We need to present our logbook records so they can contribute to public and scholarly understanding of the period.

The transcribed and edited logbook records are now a major component of naval-history.net, where they are described as:

British warship log books of the World War 1 era, totalling some 300,000 pages. The logs of over 300 ships have been transcribed, and most are online. They include coverage of Battle of the Falklands, Northern Patrol, Dardanelles, East Africa, trans-Atlantic convoys, Indian Ocean, China Station, amounting to some 60-70 percent of all major warship movements 1914-18, outside of British home waters.

But they are not enough on their own, we should combine them with other sources of information. Naval-History.Net has prepared for the centenary for some years, using contemporary sources where possible and more recent research where available. Current projects include:

  1. Chronology providing the political and military background to the war at sea.
  2. Naval Operations by Corbett and Newbolt – many of the excellent plans are online including all the Battle of Jutland – and the three volume Merchant Navy histories by Hurd.
  3. Navy despatches and relevant Army despatches from the London Gazette. Also Royal Navy honours and gallantry awards by award and by Gazette date. Includes Medal index/database by name.
  4. Royal Navy and Royal Marine casualties (researched by Don Kindell working with the Naval Historical Branch (MOD)), as well as those of the Dominion Navies and U.S. Navy & Marine Corps.
  5. Royal Navy warships and auxiliaries from the invaluable “Ships of the Royal Navy 1914-1919″ by Dittmar and Colledge. Although still in progress, all warships and many of the auxiliaries are listed by name and by type/class.

The honourable work of data rescue

Gil Compo (centre) accepts the honourable mention oldWeather was awarded in the The 2013 International Data Rescue Award in the Geosciences, from organisers Kerstin Lehnert (IEDA, left) and Bethan Keall (Elsevier, right).

Gil Compo (centre) accepts the honourable mention for oldWeather from the 2013 International Data Rescue Award in the Geosciences, with organisers Kerstin Lehnert (IEDA, left) and Bethan Keall (Elsevier, right).

One of the fun parts of working as a scientist is going to conferences, and in the geosciences, conferences don’t come much bigger than AGU. The American Geophysical Union’s 46th annual Fall Meeting ran last week in San Francisco, and it brought together more than 22,000 scientists for a week of presentations, discussions, celebrations, and beer.

Our man at AGU this year was Gil Compo, and he represented oldWeather at an important side event: The prize ceremony for the 2013 International Data Rescue Award in the Geosciences. We didn’t quite win this prize (the winner was the excellent Nimbus Data Rescue Project), but the judges liked us a lot, and we were awarded an honourable mention. So well done to all the oldWeather participants on a further well-deserved honour, and thanks to the award sponsors and organisers.

Every scientist’s must-have accessory, at any large conference or meeting, is a poster: This is a large sheet of paper (typically A0, or about 4′ by 3′) covered with artistically arranged images and results from your project, which you attach to a wall or display board, and use as a visual aid. Kevin made an excellent poster for us, combining images from all aspects of the project. You can see it on display in the background of the photo above, and if you’d like your own copy, it’s on our resources page.

Archives Update: 300,000 pages and Counting

As the year 2013 comes to a close, we at the U.S. National Archives (NARA) are pleased to announce that we have just finished imaging 300,000 pages of historic Navy deck logs and logs of the U.S. Revenue Cutter and Coast Guard vessels since the start of the NARA/NOAA collaboration in the summer of 2012. Thus we have surpassed our initial pilot project objective to digitize an estimated 250,000 logbook pages of U.S. vessels that sailed the Alaska-Arctic region. The collaborative effort among NOAA, NARA, and Old Weather was prominently featured in the September, 2013 issue of the Discover magazine.

We are a crew of three here at the Archives. At the helm is Mark Mollan, Navy/Maritime Reference Archivist, who is expertly guiding us through a sea of logbooks in various conditions and formats. Gina Kim Perry, who came onboard six months ago, is manning the station at the Digital Imaging Center at Archives 1 in Washington, DC. Our newest member is Helen MacDiarmid, who recently joined us to work at the imaging station at Archives 2 in College Park, Maryland.

Our overarching goal is to provide the best possible images for use by citizen scientist transcribers at the Old Weather project and for the general public at the NARA website. To that end, we have improved our workflow, and with expected equipment upgrades in the coming year, we hope to become even more efficient. In addition, with the start of the new year, Gina and Helen look forward to spending some of their time on outreach and education activities, with their initial focus on getting more people involved as citizen scientist transcribers for Old Weather.

If any citizen scientists will be visiting the Washington, DC, area, please look us up. We would love to meet you.

Until next time,
Gina at Archives 1

Remember the Royal Navy

The naval history side of Old Weather is making tremendous progress. Our original set of logs covered Royal Navy warships and auxiliaries of the World War 1 era, a total of 317 ships of all types. We completed the transcription of these last year, but a dedicated group of volunteer editors are developing our transcriptions into full ship histories. Of these, 264 are formatted and online, 115 have been edited and a further 30 are being edited, which means we are close to editing 50 percent of the total. Bear in mind that some of the files are up to 4Mb in size, over 400 pages, and you will realise how much effort the editors are putting in, and of a standard that to my mind is simply professional.

Another way of looking at the scale of our work on the WW1 RN logs, is that of all the major British warships and auxiliaries – capital ships, cruisers, aircraft carriers, monitors, armed merchant cruisers – that served outside British waters from 1914 to 1918, the logs cover some 70 percent of movements.

Normally the RN ships are listed by name or by type, but the following list – still in preparation, gives some idea of just how much of World War 1 at sea outside British waters is covered including major campaigns, battles and actions, as well as the run up to War and after the armistice.

To my mind, of particular importance are the China Station ships (I imagine of great interest to modern Chinese historians), the patrols of the many armed merchant cruisers and their subsequent convoy escort duties in the WW1 equivalent of the 1939-45 Battle of the Atlantic, and the ships taking part in the various interventions against the Russian Bolsheviks in North Russia, the Baltic, Black Sea and Siberia in 1918-19.

Gordon


Pre-World War 1


British Isles waters,
English Channel Fleet


Canopus, Crescent

West and Central
Atlantic


Cornwall. Hermione

Cape Station


Astraea

South America


Glasgow

Mediterranean


Blenheim, Cornwall, Vindictive (1), Weymouth

East Indies Station,
including Persian Gulf


Dartmouth, Fox

China Station


Bramble, Cadmus, Kennet, Merlin (1), Otter, Ribble, Rosario, Thistle (1), Usk, Welland, Yarmouth

Pacific


Pyramus, Psyche, Sealark

Delivery of Australian
submarines AE.1 and 2 to Australia


Eclipse, Sydney (RAN)


World War 1


BRITISH HOME WATERS and NORTH SEA

Grand Fleet, including
Harwich Force, North Sea sweeps, Norwegian convoys, shake-down cruises etc


Acacia, Achilles, Albemarle, Canterbury, Commonwealth, Devonshire, Inconstant, Invincible, Leviathan, Phaeton, Princess Royal, Renown, Warrior (1), Weymouth, Yarmouth

Battle of Heligoland
Bight, 28 Aug 1914


Invincible

Battle of Dogger Bank,
24 Jan 1915


Princess Royal

10th Cruiser Squadron,
Northern Patrol, old cruisers


Crescent,

10th CS, Armed Merchant
Cruisers


Alcantara, Almanzora, Alsatian, Andes, Arlanza, Artois, Avenger, Changuinola, Columbella, Dryad, India, Kildonan Castle, Mantua, Motagua, Orcoma, Orotava, Orvieto, Otway, Patuca, Patia, Teutonic, Victorian, Virginian; believed to include trawler Tenby Castle

11th CS, Irish waters


Minerva, Venus

British waters,
including Dover Patrol (German destroyer raid), Auxiliary Patrol, convoy
escort, guardship


Amazon, Atalanta, Attack, Attentive, Jupiter, Parramatta, Sapphire, Tenby Castle, Thistle (2)

Zeebrugge Raid


Attentive

Minelaying, British
waters


Orvieto

Kite Balloon ship
operations


Patuca

Single ship action v
Leopard 1917


Achilles,


ARCTIC

Arctic/North Russia


Acacia, Albemarle, Arlanza, Attentive, Cochrane, Cockchafer, Glory, Jupiter, Teutonic, Vindictive (1)


NORTH AMERICA and ATLANTIC

North America &
West Indies Station


Achilles, Antrim, Bristol, Cochrane, Cumberland, Devonshire, Drake, Glory, Leviathan, Laurentic, Princess Royal, Warrior (2) (during flu epidemic), Weymouth

North Atlantic convoys
(1917-18)


Achilles, Almanzora, Alsatian, Andes, Antrim, Arlanza, Artois, Bayano, Carrigan Head, Cochrane, Columbella, Cornwall, Coronado, Cumberland, Devonshire, Drake, Kildonan Castle, Knight Templar, Lepanto, Leviathan, Naneric, Orama, Orcoma, Orvieto, Otranto, Patuca, Teutonic, Victorian, Virginian

Halifax explosion, 6
Dec 1917


Highflyer, Knight Templar,

Central and
Mid-Atlantic patrols, later convoy escort, incl Canaries, Cape Verdes, Sierra
Leone


Almanzora, Argonaut, Avenger, Britannia, Canopus, Celtic, Cornwall, Empress of Britain, Kent, Kildonan Castle, Mantua, Marmora, Minerva, Motagua, Ophir, Orama, Orotava, Otranto, Patia, Victorian

West Coast of Africa,
incl. German Cameroons campaign


Astraea, Laurentic, Rinaldo, Thistle (1)

SE coast of America
Station, incl. South Atlantic patrols and later, convoy escort


Africa, Bristol, Britannia, Canopus, Celtic, Dartmouth, Kent, Orama, Orvieto, Otranto, Vindictive (1)

German South West
Africa campaign


Astraea,

Battle of the
Falklands, 8 Dec 1914


Bristol, (Canopus), Carnarvon, Cornwall, Glasgow, Inflexible, Invincible, Kent, Macedonia


MEDITERRANEAN

Mediterranean, incl.
convoy escort


Acacia, Africa, Bacchus, Blenheim, Bristol, Empress, Exmouth, Euryalus, Glory, Jupiter, Kennet, Mantis, Minerva, Raven II, Ribble, Sapphire, Usk, Welland, Wonganella

Adriatic Force and
Italian Fleet


Bristol, Britannia, Sapphire

Aegean, incl
Dardanelles and Gallipoli


Albion, Amethyst, Ark Royal, Bacchus, Ben-My-Chree, Blenheim, Canopus, Colne, Chatham, Dartmouth, Empress, Euryalus, Glory, Goliath, Grafton, Hibernia, Implacable, Inflexible, Jed, Kennet, Minerva, Phaeton, Pyramus, Ribble, Sapphire, Usk, Vengeance, Welland

Egyptian waters


Empress, Venus

Defence of Suez Canal,
to 4 Feb


Clio, Himalaya, Minerva, Proserpine, Swiftsure


EAST INDIES STATION

East Indies Station,
incl Palestine campaign, Persian Gulf , German East Africa campaign, Indian
Ocean escort


Aphis, Astraea, Ben-My-Chree, Bramble, Cadmus, Cornwall, Dartmouth, Enterprise, Exmouth, Euryalus, Fox, Jupiter, Kent, Laurentic, Manica, Mantis, Mersey, Minerva, Northbrook, Odin, Ophir, Pyramus, Raven II, Rinaldo, Sapphire, Severn, Thistle (1), Trent, Venus, Weymouth,

Single ship action v
Königsberg


Mersey, Severn

Mesopotamian Campaign


Mantis, Odin,


CHINA STATION and PACIFIC

China Station


Bee, Cadmus, Cornwall, Euryalus, Kennet, Kinsha, Laurentic, Ophir, Otter, Rosario, TB.37, Teal,, Thistle (1), Usk, Venus, Virago, Welland, Whiting, Woodcock, Woodlark

Pacific, incl SW
Pacific and coast of Americas


Algerine, Kent, Ophir, Orama, Otranto, Pyramus, Una (RAN)

Battle of Coronel


Glasgow, Otranto


MISCELLANEOUS CATEGORIES

Major ship damage or
sinking reported in logs


Albemarle, Knight Templar, Motagua, Orama

Logs covering most or
all World War 1 Service


Ark Royal, Bayano, Bramble, Carrigan Head, Coronado, Knight Templar,





post-War

Repatriating POWs


Trent

British Isles waters or
in dock


Calliope, Caradoc, Erebus, Foyle, Weymouth, Yarmouth

Russian Intervention:



North Russian
Expeditionary Force


Bacchus, Cicala, Cockchafer, Cricket, Erebus, Glowworm, M.23, M.24, M.25, M.31, M.33, Mantis

Baltic Sea


Calypso, Caradoc, Dauntless, Delhi, Dragon, Empress, Erebus, Fortol, Vindictive (2)

Black Sea


Ark Royal, Canterbury, M.29,

Siberia


Kent,

Rumania/Danube River


Aphis, Severn

Atlantic Fleet


Barham, Coventry, Delhi, Dragon, Malaya, Valiant, Warspite

North America &
West Indies Station


Calcutta (including assistance to wrecked cruiser
Raleigh),
Cornwall, Dauntless, Fortol, Mutine

South America Station


Bristol, Dartmouth, Petersfield, Southampton

West Africa


Astraea, Thistle (1)

African Station


Birmingham, Dublin, Lowestoft

Mediterranean,
including Aegean


Canterbury, Merlin (1)

Somaliland Campaign


Ark Royal

East Indies Station,
incl Persian Gulf


Ark Royal, Bramble, Cairo, Caroline, Iroquois, Mantis, Merlin (1), Moth, Odin, Rapidol, Yarmouth

China Station, incl
Yangste River


Alacrity, Bee, Cadmus, Cairo, Carlisle, Cicala, Cockchafer, Cricket, Diomede, Hawkins, Hollyhock, Iroquois, Kent, Kinsha, Magnolia, Mantis, Merlin (1), Moth, Petersfield, Scarab, Tarantula, Teal, Titania, Virago, Whiting, Woodcock, Woodlark

Pacific


Una (RAN)

Round-the-World Tours
(or in part) or Voyages


Malaya, New Zealand, Renown, Temeraire (as training ship), Warrego (RAN)

Scientific progress goes …

The oldWeather distributed supercomputer

Nobody succeeds alone, and that’s doubly true of oldWeather: not only are we legion in ourselves – a community of thousands working on logbook weather, but even as a project we are embedded in a community – we have friends and relations.

Our close relations, of course, are the other Zooniverse projects: That’s a diverse family – from the paterfamilias to the newest member, united by shared principles and the talents of the core team. But we also have more distant relatives. oldWeather is neither the first, nor the biggest, climate and weather citizen science project. climateprediction.net (CPDN) turned ten this year, and they have a very different way of doing science.

Many of the experiments climate scientists would like to do are impossible in practice: What would happen to the weather, for example, if we were to induce artificial volcanoes as a way to cool the planet? To investigate these questions, we do simulations – we build computer models of the climate system and do the experiment in the model. We have learned an enormous amount by doing this, but it does take a lot of computer time. CPDN asks volunteers to let their desktop computers contribute to this work – most of the time we use only a small fraction of the power of our computers, so this work can be done entirely in your computer’s spare time – it does not interfere with your normal use.

CPDN is also part of a family: There are lots of volunteer computing projects sharing the infrastructure provided by the Berkeley Open Infrastructure for Network Computing (BOINC) and you can contribute to any you choose.

Several of the oldWeather community have doubled their efficiency by doing citizen science and volunteer computing simultaneously: while the people are reading logbooks, their computers are simulating the climate, or Neutron stars, or malaria, or the Milky Way, or … I’d like to congratulate the oldWeather BOINC group on their tremendous contribution both to oldWeather and to volunteer computing.

Too low, terrain!

Cockpit of a NOAA P3.

Cockpit of a NOAA P3.

oldWeather is telling us a great deal about how the present climate is different from that of 100 years ago, but to make maximum use of that information, we also want to know exactly how the present climate is behaving. This will help us link our observed changes in surface weather to the basic physics of the ocean and atmosphere. To learn about the present climate we collect a rich and detailed set of observations from research ships, aircraft, and satellites.

Last year, Kevin was out making such measurements from a ship, on a research cruise in the Bering Strait. This field season he’s back out there, but he’s gone up in the world. For some purposes ground level is too low, and satellites are too high, and to fill this gap NOAA have two research aircraft (affectionally known as ‘Kermit’ and ‘Miss Piggy’). Kevin’s group have got some time on one of them, they are trying to “quantify the air-ice-sea interactions and lower atmospheric structure in the marginal ice zone, with the ultimate goal of being able to infer how recent reductions in sea ice extent in autumn will impact the atmosphere“.

The research aircraft is complex and well-equipped: According to Kevin “The NOAA WP-3 is instrumented like ten satellites. So we are able to collect a vast array of data from deep oceanography with AXCTD and AXBT expendables, SST and surface microwave emission (wind/waves/ice), upward/downward radiation, up to 22 thousand feet where we deploy dropsondes from above the clouds to characterize the structure of the atmosphere. On a survey we collect flight level data continuously while deploying AX instruments about every six minutes.

To do all that effectively requires close cooperation between the crew of the aircraft and the scientists – that’s Kevin’s job. He’s sent back this video to give us a taste of what it’s like. It looks exciting – they spend a lot of time travelling at 200 knots, only 200 feet off the ground, much to the distress of the auto-pilot – but it’s hard work: one flight means 8-10 hours flight time + 2 hours for briefings before and after.

See more about this mission on the NOAA website.

Follow

Get every new post delivered to your Inbox.

Join 1,306 other followers