GetHTTPDataProcedure

OK all
ive followed this thread for a while now...
ive tried each post out as its been posted
i still cant get my ctouch to read anything other than 0.0 (the drought has broken)

could someone (anybody) that has this working put up the complete code for me to play with..

cheers to all

 

You show me yours first!

 

Hey there
im new to the HTML thing
here's the CTA file

Cheers

 

Just pointers in the right direction would be appreciated...
I should be fine with it once this part is running correctly..

Cheers

 

Ok, firstly I think you need to look up in the help files about what these two lines are doing:

Code:

tokenPos := pos('<td headers="t1-rainsince9am">', subs);
copy(subs, subs, tokenpos + 30, length(subs));

Then check the contents of what the weather URL is serving up - ie if you use Firefox go to the web developer section and select 'view page source'.

You'll notice that what your code is asking to find is not there.....

 

Hi,

I know this is an old thread but I am trying to get BOM Temp, Rainfall and windspeed to appear on my wiser

I have updated Wiser to 1.30 which I though was the problem!!

I have posted the Logic below hoping someone may be able to help me!

I have also setup a SystemIO Variable "RainfallMeasurement" configured as REAL and added via Widget. This always shows 0.0

I am trying to pull through temp at the moment as rainfall is 0.0 at the moment!

Constants
WeatherURL = 'http://goo.gl/FVqfHV';

Global Variables
s: array [0..3999] of char;
tokenPos: integer;
subs: array[0..999] of char;
tdCount: integer;
LatestRainFall: real;
retryCount: integer;

Modules
once LatestRainFall > 0 then
begin
//enable the chicken
//LogMessage('Trigger the chicken');
//PulseCBusLevel("WAN to CNI", "Lighting", "Upstairs Office", 100%, 0, "0:10:00", 0%);
end;
//get the current weather observations from the Constants Section
GetHTTPData(WeatherURL);
delay(5);
retryCount := 0;
s := '';
repeat
ReadHTTPData(s);
retryCount := retryCount + 1;
delay(5);
until (s <> '') or (retryCount >= 3);
if retryCount >= 3 then
//LogMessage('no data');

if s <> '' then
begin
//to get the value of the rainfall since 9am we need to parse the data
//get the position in the data for the rainfall part of the table data
//tokenPos := pos('<td headers="t1-rainsince9am">', subs);
tokenPos := pos('<td headers="t1-temp">', subs);
copy(subs, subs, tokenpos + 30, length(subs));

//remove the remaining unwanted data
tokenpos := pos('</td>', subs);
copy(subs, subs, 1, tokenPos - 1);

//store the extracted value in a local variable
LatestRainFall := StringToReal(subs);
//LogMessage('Rainfall: ', LatestRainFall);
//WriteLn( 'LatestRainFall', LatestRainFall);
SetRealSystemIO("RainfallMeasurement", LatestRainFall);
end;

//check every 1 minutes ***change to 10 after finished testing
delay(60);

 

Hello all again - let's dredge up an old thread!,

I have noticed that parsing the site www.weather.com.au no longer works for forecast information.

Looking at the page it seems that the piced code that I use to parse the information on their site is still valid and should still work.

I parse json pages to serve up current conditions from BOM and that still works.

Does anybody use www.weather.com.au to get forecast information? Can you confirm it still works for you or does it fail like what I'm experiencing? Is their something tricky that webpages do to compromise the way gethttpdata works these days?

If anybody can recommend another site they use I will look into that too.

Thanks in advance.

EDIT: If anybody has a different way they get forecast information data into cbus (PICED, SHAC, etc.) these days could you share that too.

 

Hello all again,

I have had a bit of a brain wave - when using sites which are https rather than just http the GetHTTPdata does not work.

This may explain the reason why sites are dropping out of parsing as more sites become secure.

Soooooo my questions are:

a) does this sound viable (can anybody else confirm this please)
b) is there a fix that people may know for these sites or is it something that needs to change in PICED?

 

Last post for now about this - to answer my own questions above (yes answering yourself is a sign of insanity...!!)

a) I can confirm that PICED doesn't like https parsing.
b) I have used this site apixu.com to scrape a forecast via its API. It works well - although not as accurate as BOM. You can minimise extra info so PICED plays nicely with the array limitations it has. All is good again. BOM is still the "go to" for current observations in Australia - and for those still lost you need to parse the BOM .JSON pages to get past the array limitations of PICED, not the html page.

It would be nice if someone from tech could acknowledge the https limitation and hopefully address it; but being that I am a realist I doubt it, hence the advice above for overcoming the issue.

Cheers.

 

UPDATE: It seems apixu no longer exists. Workaround is the RSS feed from weather.com.au for your weather forecasts; everything else is going https or they broadcast radically inaccurate forecasts.......