oldWeather Arctic
Today we launch a new fleet on oldWeather.org: the focus this time is on Arctic voyages, and the logbooks are from the collection of the US National Archives in Washington. We’ve ships from the Revenue and Coast Guard, the Navy, and the Coast Survey; and they include some famous names and some exciting voyages.
The Arctic is very sensitive to climate variability and change. This year (2012) was a record year for sea-ice: there was less sea-ice this September than for any other year for which we have good satellite records. But the satellite records only go back to 1979, and we need many more than 30 years of records to really understand how the Arctic climate behaves. This means we need to rescue the weather records of the people who travelled there – to read the logs of Arctic voyages.
If you joined in the original oldWeather, you’ll notice some differences in this new version: There are fewer ships (at least to start with, we’ll be adding more regularly), but the records for each ship usually cover many years, so we have just as many pages to read. These logbooks are also older (back to 1850 in some cases), and differently laid-out, so we’ve had to change the way you enter data: Basically it’s the same – select the location on the log page with an important record and then type the record into the pop-up box – but the details have changed. So whether you’re a new recruit or an old hand, please experiment until you get used to it – there is a tutorial to guide you, and help and encouragement on the project forum.
We’re still looking for all weather records, and anything else you read and think is interesting or notable. There will be plenty of notable historical events: the dangers of sailing through the ice add a lot of drama to the stories in the logs – whether you prefer the daring rescue by USRC Bear, ice and fire on the USS Rodgers, or the so-far-unknown adventures of less famous ships.
The voyage of USRC Thetis – April to September 1884
We’ve been running the beta-test of oldWeather Arctic for several weeks now, and we’ve accumulated plenty of completed log pages – that’s log pages that have been transcribed by the three people we need to get reliable results. So it’s time to have a good look at the results we’re getting: Is our new interface collecting the transcriptions properly, and are the transcriptions we’re getting accurate and useful?
This time I’ve tried to show explicitly the link between what we’re doing on the website and the numbers that are going to the science team. The image below shows this for a single log page from USRC Thetis (click on the image for a bigger version).
The log page for USRC Thetis from May 5th 1884 – as transcribed through oldweather.org. The coloured boxes overlain on the log page are those drawn by the transcribers (blue for weather, red for locations, and grey for dates). The text on the right hand side shows the information collected from the transcriptions.
(Bigger image, Video showing all pages.)
The left hand side of the image shows what we’re doing on oldweather.org – a log page marked-up with the locations of valuable data. (This time I’ve looked only at the dates, positions, and hourly weather observations – the historical events and informal weather records (including ice observations) are just as important but I didn’t have space for them.) As each page is transcribed by at least three people, there are usually three selections for each record. The right hand side shows the values extracted from the transcriptions.
For this page it’s working very well: we’re getting the detailed weather and ship-position information we need. Of course, that’s just one page – we need to do that for every page, and that’s shown in this video, which shows the transcribed data streaming out of the log consistently and accurately. All our hard work transcribing is delivering the detailed weather records the scientists need.
We can also look at the 2718 new weather observations we’ve rescued from the Thetis so far. How do they compare with more recent observations? Were the sailors on this ship careful and accurate observers? To judge this I like to compare the oldWeather observations (red points in the figure below) with modern records.
Observations made by USRC Thetis on her voyage from New York to Smith Sound in summer 1884 (red dots), compared with modern climatologies (black lines).
The top left image shows the route of the ship: from New York up through the Labrador Sea and Baffin Bay and back – a true Arctic voyage. The bottom right image shows that measured air temperatures were typically lower on the voyage in 1884 than the average (climatology) for the last few years – an intriguing result (though there are many possible reasons for it). Top right and bottom left are air pressure and wind speed, these are harder to compare because for pressure and wind we expect bigger differences between an observation (a point value) and a climatology (an average over several years). Rather than going into details I’ll just say that I’m very pleased with these results too; this comparison is exactly what we’d expect from good-quality, useful observations.
So well done USRC Thetis and all who sail with her – both her original crew who took the observations, and the oldweather crew led by Lekiam, Jelliott8 and lollia paolina.
The transit of Venus
The weather in Exeter yesterday was best described as “ocr“, so I missed the transit this time. Fortunately, the skies were clear back in 2004, and I remember the experience of peering through a pair of binoculars equipped with a sun-filter and seeing the small black dot of Venus silhouetted against the sun.
The transit of Venus is a periodic event, and the big year was in 1769. I understand that the astronomers valued the transit as a way to get a handle on the size of the universe; but the real virtue was that it provided an excuse for the British Government to send an expedition down into the South Pacific. That expedition was commanded by James Cook, and it started the career of the greatest explorer of them all.
Of course, as a naval officer (Lieutenant, at the time), Cook kept a journal. If your eyesight is up to it, you can read his account of June 3rd 1769 in the original handwriting; but I admit that I looked up Project Gutenberg’s transcription:
Saturday, 3rd. This day proved as favourable to our purpose as we could wish. Not a Cloud was to be seen the whole day, and the Air was perfectly Clear, so that we had every advantage we could desire in observing the whole of the Passage of the planet Venus over the Sun’s Disk. We very distinctly saw an Atmosphere or Dusky shade round the body of the planet, which very much disturbed the times of the Contact, particularly the two internal ones. Dr. Solander observed as well as Mr. Green and myself, and we differ’d from one another in Observing the times of the Contact much more than could be expected. Mr. Green’s Telescope and mine where of the same Magnifying power, but that of the Doctor was greater than ours. It was nearly calm the whole day, and the Thermometer Exposed to the Sun about the Middle of the day rose to a degree of heat we have not before met with.
The ideal weather observer does not expose his thermometer to the sun (shade temperatures please), so perhaps it’s no great loss that Cook’s journal does not contain regular weather observations. For those, we must turn to the Master’s log of HM Bark Endeavour (the closest equivalent to the familiar modern-day logs). For the day of the transit, this records “Little wind and variable with fine pleasant clear weather”. I reckon that’s “Lt. Airs, Var., 1, b” in our notation. Sadly, none of the logs contain regular thermometer or barometer observations – Cook did better on his subsequent voyages – but we do get wind speed and direction reports for every day.
Analysis of the oldWeather data
Now that we’ve completed the original batch of logs we were working on in oldWeather, we’ve started to release the new weather observations recovered for use in scientific investigations. Leading the way in such investigations is Ed Hawkins of the University of Reading, who’s started a series of posts on his blog describing the value of oldWeather for the study of Arctic climate and sea-ice, the North Atlantic Oscillation, and reducing uncertainties in Atlantic pressure fields.
oldWeather Credits Reel
One of the main uses of the weather observations that we are collecting is in new reanalyses – reconstructions of weather and climate over the last few decades or centuries. This week, dozens of scientists working on weather and climate reconstruction are meeting for a workshop on reanalyses and historical weather observations, hosted by the Royal Netherlands Meteorological Institute at De Bildt.
This is an opportunity to tell everybody working in the field just how much we’ve achieved with oldWeather over the last 11 months, so I’m giving a presentation highlighting our results. As you’ll have seen from earlier blog entries, there’s plenty to present – so my biggest challenge is in working out how to give credit to all the project participants: It’s a firm rule in science that you should credit all your collaborators in any project, but there are 9566 people who’ve made a significant contribution to oldWeather (at the last count). So to list them all I’ve borrowed a technique from the movies, and made a credits video – this video is being premiered at the meeting (as part of my talk).
Of course it’s not enough just to have lots of people involved, we’ve also got to generate lots of new scientific results. So I’ll also be showing another video – less detailed, but faster and much more colourful – showing the 841848 new weather observations that we’ve generated.
Very old weather.
Edmond Halley is best known for his comet, but he was one of the great polymaths – as well as making astronomical discoveries he was also a notable meteorologist: he did important early work understanding the trade winds and monsoons. It’s less well known that that he was also a Naval Officer: in 1699 he was granted a commission as captain in the Royal Navy, and he commanded HMS Paramour (a pink) on an expedition into the South Atlantic to investigate the variation of the compass.
His main concern was with magnetism, but as a man of wide interests, Halley took with him examples of those two exciting modern scientific instruments: the thermometer and the barometer. I can’t find the logbook of the voyage, but Halley’s notes have survived: they were published by Alexander Dalyrmple, in 1775, as part of “A collection of voyages chiefly in the Southern Atlantick Ocean“. They date from 220 years before the logbooks we’re used to in OldWeather, but to anyone who’s looked at our logbooks they are oddly familiar: records of latitude, longitude, wind force and direction and, in the left-hand margin, thermometer and barometer readings.
In 1699 the barometer had been around for more than 50 years, and the barometer records in Halley’s account are clearly in the familiar inches of mercury. But the thermometer did not become a reliable, precision instrument until about 1725, when Fahrenheit invented the mercury thermometer with a standardized, calibrated scale. So when Halley says the temperature is ’33′ it’s not immediately obvious how this should be interpreted. Careful scholarship has established, however, that Halley was using a thermometer designed by Robert Hooke, and lavishly described in his book Micrographia:
The Stems I use for them are very thick, straight, and even Pipes of Glass [...] above four feet long [...] [filled] with the best rectified Spirit of Wine highly tinged with the lovely colour of Cocheneel, which I deepen the more by pouring some drops of common Spirit of Urine, which must not be too well rectified, [...]
From Hooke’s description we can convert Halley’s reported units into modern equivalents at least approximately – Halley’s ’33′ was about 8°C.
The diary entries are mostly routine accounts of the movements of the ship, but occasionally he puts in longer and more interesting reports: here’s an example from Thursday 1st February 1700, when they were close to South Georgia, in the cold waters of the Southern Ocean:
[...] between 4 and 5 we were fair by three Islands as they then appeared; being all flat on the top, and covered with Snow milk white, with perpendicular Cliffs all round them [...] The great height of them made us conclude them land, but there was no-appearance of any tree or green thing on them, but the Cliffs as well as the tops were very white, our people called A by the name of Beachy-Head, which it resembled in form and colour. And the Island B in all respects was very like the land of the North-foreland in Kent, and was at least as high and not less than 5 miles in front, [...]
The following day they were disconcerted to discover that these ‘islands’ had moved, and fled north to warmer waters. This is the first recorded sighting of a tabular iceberg.
Halley’s observations are probably not of great value to climate scientists: his instruments were state-of-the-art for 1699, but it took decades longer for such observations to became accurate and plentiful enough for climate reconstructions. He did set a precedent though – possibly as the first person to go to sea with a barometer and a thermometer – and we’re still following his example more than 300 years later.
0.02953 inches to the hectopascal
When scientists talk about pressure, they measure it in Pascals (Pa: the SI unit for pressure). For atmospheric pressure, 1Pa is an inconveniently small number, so we lump them together in groups of 100 and talk about hectopascals (hPa: 1hPa=100Pa). The atmospheric pressure at sea level is usually given as 101325 Pa, which is approximately 1000 hPa; so 1 hectopascal is also referred to as 1 millibar – when you hear your weather forecaster talking about millibars, hectopascals are what he’s really using. The ships, however, don’t measure pressure in hectopascals or even millibars; they measure it in inches. This is an artefact of the way they measure the pressure – with a mercury barometer.
Back in the early 17th century there was much discussion among the scientists of the day about why it was impossible to pump water more than about 10m upwards. It was Evangelista Torricelli, in 1643, who realised not only that the height to which the water rose was determined by the weight of the surrounding air, but also that you could use this effect to measure changes in the air pressure. A 10m column of water is a nuisance to work with, so he switched to the much heavier mercury as his working medium, and made the first ever barometer measurement.
We’ve been measuring air pressure in the same way ever since – balance the weight of a column of mercury against the weight of the surrounding atmosphere, and the taller the column the higher the atmospheric pressure. At sea-level, the column will be about 76cm (29 inches) high, and the changes in atmospheric pressure as the weather changes cause fluctuations of up to a few inches. The pressure is proportional to the height, so we can get the pressure in hPa by multiplying the height in inches by 33.86389.
Of course, making precise measurements requires great care (very pure mercury, no air in the tube, careful calibration, …) but by our period (1914) barometer manufacturers were making very good instruments. There are, unfortunately, still a few complicating factors which we need to be aware of:
- The weight of a column of mercury changes with temperature – the weight of 760mm of mercury is less when it’s hot than when it’s cold, so we need to adjust for this when calculating pressure from height. A further complication is that the column height is usually measured using brass measuring rods, and the length of brass rods also changes with temperature. So we apply a correction from a table or an empirical formula – these tables vary slightly depending on the barometer design, but in OldWeather we don’t usually know the make of barometer in use so we use a generic table. To make this temperature correction we need, of course, to know the temperature of the barometer: Almost all mercury barometers have a thermometer attached and it is usual to record the barometer height and attached thermometer temperature together – as is done in many of our logs. Moving from 0C to 35C (Arctic to the tropics or February to July in the UK) would introduce a change of about 0.5% (2 tenths of an inch).
- The weight of a column of mercury changes with latitude. We launch satellites from French Guiana, rather than Europe, because satellites weigh less in French Guiana than they do in Europe. Moving from Plymouth to Singapore would reduce your weight by about 0.2% (about 8 hundredths of an inch)
- We generally want the pressure at sea-level. We usually keep the barometer above sea-level, so we need to add a little to the pressure to adjust for this. Every 80 or 90 feet above sea-level reduces the pressure by 1 tenth of an inch.
- It’s usual to measure the position of the top of the mercury column. As the mercury rises in the tube, the level of the mercury in the cistern at the bottom of the tube will fall. Because the mercury column balancing the atmosphere runs from the top of the level in the tube to the level in the cistern, we need to add a little to measured height changes to allow for this.
- If the glass tube containing the mercury column is narrow (to reduce weight and to damp oscillations) the height of the mercury will be reduced by capillary action. We need to add a little to the measured height to allow for this.
We call these, respectively, the temperature correction, the gravity correction, the height correction, the capacity correction and the capillary correction. By 1914, with a good barometer, the last two should have been allowed for in the instrument’s calibration and operation, and the third is small for ships, but we still need to make the first two corrections. The changes involved are small compared with the changes associated with short term weather, but they are important for correctly representing the more subtle, longer-term changes.
Mercury barometers are great for fixed, stable, weather stations. They are however expensive, difficult to read accurately in a ship in motion, a terrible nuisance to carry around, and really too fragile for service in a warship. So much ingenuity has been spent on devising cheap, portable, alternatives. The aneroid barometer is essentially a sealed metal bellows that grows and shrinks as the air pressure rises and falls, coupled to machinery to amplify its movements and display them on a scale. These first appeared in 1843, but it took a long time to make them accurate and reliable enough for serious use. By 1914, however, they were coming into use, and it’s clear from the logs that our ships used both mercury and aneroid barometers. Aneroids don’t require gravity, capacity, or capillary correction – and are mostly deliberately designed to be insensitive to temperature changes, so they don’t need an attached thermometer measurement. Nowadays aneroid barometers report pressure in hPa, but back in 1914 most gave readings in inches of mercury. So far I’ve only seen one ship reporting pressures in hPa – HMS Glowworm.
Were the aneroids on our ships less accurate than mercury barometers? more accurate? different in some subtle way? I don’t know – but I look forward to finding out. So if you see any reference in the log to the type or make of barometer in use, please transcribe it. We don’t need to know what they were using, as we can guess with good accuracy, but it does help. A few ships record both mercury and aneroid barometer readings – if you see this, please transcribe both of them; the comparison between them helps us estimate the accuracy of the measurements.
New Zealand gets its money’s worth.
Remember the fog of ignorance – the uncertainties in global weather reconstructions that our new observations will help to clear away? Here’s another view of the problem (click on the image to see the movie version).
A global weather reconstruction for midnight, May 25th, 1911. Coloured dots are surface pressure observations (blue - low pressure (storms), red - high pressure (fine weather)), the coloured lines and arrows show the sea-level pressure and 10m winds reconstructed by the twentieth century reanalysis. Areas fogged-out mark regions where reconstruction is impossible because there are no observations (yet - the OldWeather observations are not yet included in this analysis).
The skill with which we can reconstruct past weather depends critically on how many observations of it we have, and for the period we’re investigating in Oldweather, it varies a lot from place to place: in the UK and US we can do reconstructions precisely, but for much of the rest of the world – the southern hemisphere in particular – we’re still very uncertain.
So how much improvement do we expect when we add our new observations to these reconstructions? Obviously this depends on where the new observations were made, and on how good they were, so it’s interesting to compare a few, from a range of times and places. I chose to follow the battlecruiser HMS New Zealand on her circumnavigation in 1919: comparing, at every point in the voyage, her observations of the weather (air pressure) with the existing reconstruction – our best estimate of the weather before Oldweather.
Sea-level pressure experienced by HMS New Zealand during a circumnavigation in 1919. Black dots mark the observed pressures, the blue band shows the uncertainty range in pressure from the Twentieth Century reanalysis - a reconstruction using pre-existing surface pressure observations.
The New Zealand started her trip in Plymouth in February, where we already knew that the weather was miserable – the thinness of the blue line means that we already had enough nearby observations to be sure of the weather, and the spikes in the pressure series are depressions blowing through. The good news is that the ship’s observations agree almost exactly with the reconstructions using other records, which means that the New Zealand was making good observations – they’d calibrated the barometer correctly and were careful in their measurements.
Almost as soon as the ship leaves the UK, the blue line widens – our reconstructions are less certain of the weather. it also gets less variable, as they are sailing in the more stable weather of the tropics. The best illustration of the value of the new data, however, comes in the southern hemisphere: for Australia we already have some observations, so our reconstruction was fairly well constrained already, but New Zealand (the country) was deep in the fog of ignorance, and the wide blue band at that point shows the huge uncertainty in the local weather – an uncertainty that we’re now able to remove using the new observations from New Zealand (the ship).
It will be a while before we can make another global weather reconstruction that includes our new Oldweather observations (that’s a major project taking lots of supercomputer time), but plans for doing it are well advanced. When we’ve done this, and I’m able to repeat this analysis using the resulting reconstruction, then the weather will be precisely known all along the route of the ship (the blue band will be thin at all points) and New Zealand (the country) will have emerged from the fog of ignorance – it’s weather conditions will be clearly known.
Which is only fair, as the New Zealanders paid for the construction of the eponymous battlecruiser in the first place.
Wei Hai … where exactly?
The 100 most-frequently used place names in the logs. (Made with Wordle.net)
Working with the logbooks has done wonders for my knowledge of global geography. If it’s at sea level, one of our ships has probably been there, or at least mentioned sighting it on the way past, and we can travel, vicariously, with them; from Abadan to Zanzibar by way of Cockatoo Island, Fernando Po, Nuku’alofa, Surabaya, and Wuhu (with assistance from lighthouses on Mwana Mwana, Muckle Roe, and Makatumbe).
We’d expect the Royal Navy to spend most of their time in British ports, but we deliberately chose the logs we’re looking at to include those going foreign, and omit the stay-at-homes, because this gives us better information on global weather. This choice means that foreign ports are the most frequently mentioned in our logs. In the 300,000 or so log-pages we’ve looked at so far, Hong Kong tops the ‘most visited’ table (with 23,000 mentions), followed by Bermuda and Shanghai. The first UK port comes in fourth: Devonport (6000 mentions) and though most of these are for the UK naval base near Plymouth, its statistics are boosted by the existence of another base of the same name in Auckland.
The existence of two Devonports highlights a difficulty we run into in using the port names. When the ship is in port, and sometimes when it is operating close to land, the port name or landmark is the only information we have on the ship’s location. So we have to convert the name into a latitude and longitude, and this can be challenging. For many ports a position is not hard to find: Gibraltar, Bombay, Glasgow and Aden are all well known. Many more are only a quick web search away: Esquimalt is on Vancouver Island, Thursday Island is in the Torres strait, and Walvis Bay is in Namibia.
After that it gets harder – East London is nowhere near East London, St Vincent usually means Cape Verde, rather than the identically named place in the West Indies or the Portuguese headland made famous by the battle of 1797. ‘No. 10 dock’, ‘No. 5 buoy’, and ‘No. 7 warf’ are all in Plymouth, but ‘on patrol’, ‘southern base’, and ‘on surveying ground’ could be anywhere.
The Navy are renowned for their courage and seamanship. Their orthography and penmanship are a little more variable, so we have Wei Hai Wei (2345 entries), Wei-hai-wei (1357), Wei hai Wei (633), Wei hai wei (314), Wei-Hai-Wei (231), wei hai wei (91), wei lai wei (69), Weihai wei (57), wei-hai-wei (53), Wei hei wei (33), Wei-hei-wei (32), WEI HAI WEI (30), and even W.H.W (26) – all of which are references to the same place.
With the technology of 1914-22, sorting all this out into a set of positions would have been a terrible job; but modern internet search engines, atlases, encyclopaedias and gazetteers are very powerful tools for tracking down obscure and badly spelt place-names. Today I’m particularly grateful that I live in the future.
The weather in 1.85 characters.
My desk in the Met Office is some way from a window, but if I peer across the heads of a few colleagues I can see that the weather outside is, well, disappointing: A gloomy day, with the sky filled with mottled grey clouds from horizon to horizon (though at least it’s stopped raining). Here in the UK we’re famously obsessed with talking about the weather, but sailors would have no time for such waffle: Following an example set by the famous Admiral Beaufort they record the current weather in a terse code, and today’s weather in Exeter would be simply ‘o’ (overcast), or perhaps ‘oc’ (overcast cloudy) if they were feeling extravagant.
The weather code system has evolved quite a bit since Beaufort’s day, and it’s a powerful and concise way of recording notable weather events. The basic code records the amount of cloud in the sky, and ranges from ‘b’ (clear sky or mostly so), through ‘bc, and ‘c’ to ‘o’ (overcast). These are by far the most common codes, but you can add to them to record many of the various nastys the atmosphere can inflict on you – there are codes for rain, snow, hail, gales, squalls, fog etc.
This means that the longer the code recorded in a logbook, the worse the weather was (or at least the more exciting it was). The longest code I’ve found in the logs completed so far is ‘ocpqrlt’ (overcast, clouds, showers, squalls, rain, thunder and lightning) from HMS Bacchante, at Dakar at midnight on 31st August 1917. (Thanks to captain richbr15, lieutenant dazedandconfused, and the crew for patiently typing all that in). This sort of detail, however, is rarely necessary, and, on average, the logs only need 1.85 characters to record the current weather.
I’m excited by the weather codes because they offer a new opportunity to test our climate models. In principle, if we know the surface pressure and temperature (also in the logs, of course) our models should tell us where it’s clear, where it’s cloudy, where it’s raining, and even about thunderstorms and squalls. In practice it’s not quite as easy as that, partly because our computers are not yet powerful enough to run atmosphere models that are detailed enough to resolve small features like thunderstorms and squalls; but even so I look forward to learning more about the accuracy of our cloud and rainfall models. So please keep entering the weather codes – we need the ordinary records of cloud cover as well as the unusual events.
Since I started writing this the rain has come back, so I should modify my current weather report to ‘or’; but improvement is in sight – the forecast for this weekend is for ‘bc’ (broken cloud), maybe even ‘b’ (little or no cloud) at times. The designers of the weather codes were uninterested in particularly fine weather, so there’s no way of encoding ‘glorious sunshine’ for example (‘gs’ would be gales and snow). Still I wish you all as much ‘b’ as you care for, except for a dose of ‘r’ (rain) for anybody praying for it.
