GetHTTPDataProcedure

Discussion in 'Pascal Logic Code Examples' started by DoctorZoidberg, Aug 18, 2010.

  1. DoctorZoidberg

    Newb

    Joined:
    Aug 6, 2012
    Messages:
    3
    Likes Received:
    0
    Location:
    Melbourne
    OK all
    ive followed this thread for a while now...
    ive tried each post out as its been posted
    i still cant get my ctouch to read anything other than 0.0 (the drought has broken)

    could someone (anybody) that has this working put up the complete code for me to play with..

    cheers to all
    :)
     
    Newb, Aug 6, 2012
    #21
    1. Advertisements

  2. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    You show me yours first!
     
    Ambro, Aug 7, 2012
    #22
    1. Advertisements

  3. DoctorZoidberg

    Newb

    Joined:
    Aug 6, 2012
    Messages:
    3
    Likes Received:
    0
    Location:
    Melbourne
    Hey there
    im new to the HTML thing
    here's the CTA file

    Cheers
    ;)
     

    Attached Files:

    Newb, Aug 7, 2012
    #23
  4. DoctorZoidberg

    Newb

    Joined:
    Aug 6, 2012
    Messages:
    3
    Likes Received:
    0
    Location:
    Melbourne
    Just pointers in the right direction would be appreciated...
    I should be fine with it once this part is running correctly..

    Cheers
     
    Newb, Aug 7, 2012
    #24
  5. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    Ok, firstly I think you need to look up in the help files about what these two lines are doing:

    Code:
    tokenPos := pos('<td headers="t1-rainsince9am">', subs);
    copy(subs, subs, tokenpos + 30, length(subs));
    Then check the contents of what the weather URL is serving up - ie if you use Firefox go to the web developer section and select 'view page source'.

    You'll notice that what your code is asking to find is not there.....
     
    Ambro, Aug 7, 2012
    #25
  6. DoctorZoidberg

    dsb

    Joined:
    Mar 19, 2016
    Messages:
    1
    Likes Received:
    0
    Location:
    Australia
    Hi,

    I know this is an old thread but I am trying to get BOM Temp, Rainfall and windspeed to appear on my wiser

    I have updated Wiser to 1.30 which I though was the problem!!

    I have posted the Logic below hoping someone may be able to help me!

    I have also setup a SystemIO Variable "RainfallMeasurement" configured as REAL and added via Widget. This always shows 0.0

    I am trying to pull through temp at the moment as rainfall is 0.0 at the moment!

    Constants
    WeatherURL = 'http://goo.gl/FVqfHV';

    Global Variables
    s: array [0..3999] of char;
    tokenPos: integer;
    subs: array[0..999] of char;
    tdCount: integer;
    LatestRainFall: real;
    retryCount: integer;

    Modules
    once LatestRainFall > 0 then
    begin
    //enable the chicken
    //LogMessage('Trigger the chicken');
    //PulseCBusLevel("WAN to CNI", "Lighting", "Upstairs Office", 100%, 0, "0:10:00", 0%);
    end;
    //get the current weather observations from the Constants Section
    GetHTTPData(WeatherURL);
    delay(5);
    retryCount := 0;
    s := '';
    repeat
    ReadHTTPData(s);
    retryCount := retryCount + 1;
    delay(5);
    until (s <> '') or (retryCount >= 3);
    if retryCount >= 3 then
    //LogMessage('no data');

    if s <> '' then
    begin
    //to get the value of the rainfall since 9am we need to parse the data
    //get the position in the data for the rainfall part of the table data
    //tokenPos := pos('<td headers="t1-rainsince9am">', subs);
    tokenPos := pos('<td headers="t1-temp">', subs);
    copy(subs, subs, tokenpos + 30, length(subs));

    //remove the remaining unwanted data
    tokenpos := pos('</td>', subs);
    copy(subs, subs, 1, tokenPos - 1);

    //store the extracted value in a local variable
    LatestRainFall := StringToReal(subs);
    //LogMessage('Rainfall: ', LatestRainFall);
    //WriteLn( 'LatestRainFall', LatestRainFall);
    SetRealSystemIO("RainfallMeasurement", LatestRainFall);
    end;

    //check every 1 minutes ***change to 10 after finished testing
    delay(60);
     
    dsb, Mar 20, 2016
    #26
  7. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    Hello all again - let's dredge up an old thread!,

    I have noticed that parsing the site www.weather.com.au no longer works for forecast information.

    Looking at the page it seems that the piced code that I use to parse the information on their site is still valid and should still work.

    I parse json pages to serve up current conditions from BOM and that still works.

    Does anybody use www.weather.com.au to get forecast information? Can you confirm it still works for you or does it fail like what I'm experiencing? Is their something tricky that webpages do to compromise the way gethttpdata works these days?

    If anybody can recommend another site they use I will look into that too.

    Thanks in advance.

    EDIT: If anybody has a different way they get forecast information data into cbus (PICED, SHAC, etc.) these days could you share that too.
     
    Ambro, Jan 27, 2019
    #27
  8. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    Hello all again,

    I have had a bit of a brain wave - when using sites which are https rather than just http the GetHTTPdata does not work.

    This may explain the reason why sites are dropping out of parsing as more sites become secure.

    Soooooo my questions are:

    a) does this sound viable (can anybody else confirm this please)
    b) is there a fix that people may know for these sites or is it something that needs to change in PICED?
     
    Ambro, Apr 3, 2019
    #28
  9. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    Last post for now about this - to answer my own questions above (yes answering yourself is a sign of insanity...!!)

    a) I can confirm that PICED doesn't like https parsing.
    b) I have used this site apixu.com to scrape a forecast via its API. It works well - although not as accurate as BOM. You can minimise extra info so PICED plays nicely with the array limitations it has. All is good again. BOM is still the "go to" for current observations in Australia - and for those still lost you need to parse the BOM .JSON pages to get past the array limitations of PICED, not the html page.

    It would be nice if someone from tech could acknowledge the https limitation and hopefully address it; but being that I am a realist I doubt it, hence the advice above for overcoming the issue.

    Cheers.
     
    Ambro, Apr 3, 2019
    #29
  10. DoctorZoidberg

    Ambro

    Joined:
    Nov 23, 2010
    Messages:
    101
    Likes Received:
    1
    Location:
    Adelaide
    UPDATE: It seems apixu no longer exists. Workaround is the RSS feed from weather.com.au for your weather forecasts; everything else is going https or they broadcast radically inaccurate forecasts.......
     
    Ambro, Nov 21, 2019
    #30
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.
Similar Threads
There are no similar threads yet.
Loading...