I have written several command-line weather programs for different services -- many of which I had to abandon. For example, I wrote a popular one for Weather Underground that I had to retire because WU stopped allowing free access to their API.
My latest effort is one that just deals with NWS/NOAA directly, and reading this article has me wondering if part of their bandwidth trouble is a product of the way their API works.
The API is a huge set of linked JSON documents. So it very often happens that a request for data is actually half-a-dozen requests through embedded URLs (all of which are made by the client one after the other). It's all very logical in some ways, but at the same time a bit byzantine. I often wonder which entity is served by the way it's set up. From my perspective, it's easily the hardest API to work with. Weather Underground was wonderful, and so was Dark Sky (which I played with for awhile).
I've had people over the years ask if one or another of my tools could be used for large scale or very rapid weather data collection, but all of the services I have worked with throttle connections exactly as NOAA is proposing. I suppose I'm at least relieved to hear that they're not restricting access by independent developers!
You seem to know a lot about weather, so apologies if this comes off as a stupid question.
What value add do companies such as AccuWeather and others provide over simply going to weather.gov? Is their UI simply better, or do they run their own weather models? Thanks
Companies very rarely run their own weather models. This is because the models require a ton of computing power. Ex: the USAF current weather model is run on "Thor"[0][1]. It's a .9 PFlop system. And it uses approximately all of it, on a consistent basis. If it didn't, they'd increase the model resolution to match, or run the model more often to incorporate observed data faster.
Most of the companies like AccuWeather et al provide, is better UI, and tailored weather. Not just there's going to be a severe thunderstorm, but "these truck routes are going to be affected by weather, so expect them to be 40% slower than normal" or similar.
I hate wunderground - it is a giant, bloated mess of a website with a UI that jumps all over the place ...
However, I can't find anyone else to give me a 10 day forecast. It seems like wunderground is the only place that collates it like that, which I like very much.
Am I wrong ? Is there some other source for a proper, detailed, 10 day forecast ?
I just use https://www.weather.gov/ because its not bloated. Unfortunately it doesn't have 10 day forecasts and I believe the reason they don't show 10 day forecasts is because they are not very accurate. That far out your better off looking at forecast ensembles and looking at what the Euro, Canadian, as well as the GFS model is forecasting. But no common, free weather websites I'm aware of exposes that info. If you live in the west you can use this site to a degree[1], although its more targeted to snow/ski forecasting.
Is it really difficult to find 10 day forecast? Windy is a particularly good app for these kinds of things.
I also run a small REST API service (oikolab.com) that provides location-based forecast data out to 10 days using NCEP service. I started out using DarkSky and Wunderground a couple of years ago but couldn't really get the type & quality of data I needed so I ended up creating one from the original raw data.
It is a 're-packager' in a sense but what we bring to the table is the ability to query the data in a time-series manner efficiently for specific set of end-users. The way NCEP's GRIB data is packaged makes it very slow and painful to do this if you wanted weather parameters to feed into other analytical models (e.g. building energy simulation, time-series forecast etc).
Im in the same boat, their site performance has seem to just been getting worse over time and it is barely even functional on my phone, however I don't know of anyone else that condenses almost the entirety of a highly detailed 7+ day weather report into such a small and useful little graph with every detail you would want. What would take multiple pages worth of reporting is covered in a small graph the size of my hand.
It was invaluable when I did outdoor work, everyone else was making guesses and extrapolations based on the shitty reports from different local news stations and radar and satellite maps, meanwhile I could tell exactly when a big storm would role through and if rain would be spotty or temperature swings with just a glance for multiple days ahead.
Look for an app which lets you download GRIB files. PocketGrib and LuckGrib are a couple to try. You can download GFS data (produced by NOAA) for the area/timescale/data you are interested in, drop a cursor anywhere on the map and view a meteogram.
This is the same data used by "value add" weather forecasting sites and services to produce their forecasts.
Accuweather will give you a 3 month forecast! I had to blink a few times to make sure I got that right. Their existing long term forecast goes to March 10, 2021.
It's worth being aware of its shortcomings, as even its 25 day forecast has been shown to be no more accurate than climatological averages.[1] They also have done some shifty stuff in the past regarding data privacy, among some other weird things.[2]
I have a degree in meteorology. I can forecast weather for the 4th of July over next 3 years (2021, 2022, and 2023). here in Chicago, Illinois.
will be upper 80s, low 90s. Chance of thunderstorms in afternoon.
Congrats, i have now predicted a typical 4th of July and will be correct more than normal. Damn i should have built a web company and sold my ideas for a bajillion dollars
But I still appreciate that the Dark Sky feature isn't artificially limited to a particular date. They have a nice error bar on the temperature that gives you an indication of how much it's unknown.
There is a text box in the top left which says "Local forecast by "City, St" or Zip Code". Enter your city, state, or zip code in that box and select the location. Then it opens the forecast page. I have included a link to San Francisco below. You get a full 10 day forecast. I'm not sure what else you want.
You must be counting the day and night forecasts as separate days. There are no 10-day forecasts on the linked page. The furthest-out forecasts are 7 days into the future.
A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations.https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
The article mentions rate limiting by connections:
> limit users to 60 connections per minute
You can make multiple requests over a single connection.
> allow servers to instruct clients on how to construct appropriate URIs
Right. That would be great, and actually, that's what the commercial services do.
With NWS it goes something like this: Say the user has a lat/long. You fire off an http request for a document that tells you what "quadrant" you are in and the URL for it. You then fire off a request for the quadrant document, that lists the products that are available (and their associated URLs). You then fire another request for the actual product for that quadrant . . .
This is not a real example, but this is basically how it works. Even when you can construct a single URL with the info you need, you still need to make multiple requests to have enough information to build the URL (if that makes sense).
> You can make multiple requests over a single connection.
You mean with some kind of http keep-alive thing? I'm not sure I can do that in these circumstance (with cURL).
My latest effort is one that just deals with NWS/NOAA directly, and reading this article has me wondering if part of their bandwidth trouble is a product of the way their API works.
The API is a huge set of linked JSON documents. So it very often happens that a request for data is actually half-a-dozen requests through embedded URLs (all of which are made by the client one after the other). It's all very logical in some ways, but at the same time a bit byzantine. I often wonder which entity is served by the way it's set up. From my perspective, it's easily the hardest API to work with. Weather Underground was wonderful, and so was Dark Sky (which I played with for awhile).
I've had people over the years ask if one or another of my tools could be used for large scale or very rapid weather data collection, but all of the services I have worked with throttle connections exactly as NOAA is proposing. I suppose I'm at least relieved to hear that they're not restricting access by independent developers!