What is the best way to handle PyON data with AutoIt3?

[mikkokh]

Hi.

Does anyone know how to parse PyON formated data (received from Folding@home Client) to format that AutoIt3 supports?
At now, I use many clumsy stringregexp() queries to get all values that I want to get.

Is there a better way?

[Jos]

12 minutes ago, mikkokh said:

PyON formated data

Which is what exactly?

Jos

[mikkokh]

25 minutes ago, Jos said:

Which is what exactly?

Jos

Your guess is pretty much as good as mine...  Decide by name and what it looks like, I would say that it is a JSON substitute for Python.

It's mentioned many times when I give a help command for FAHClient (see bottom).

expandcollapse popup

auth                        Authenticate.
  error                       Error message.
  exit                        Exit the command processor
  heartbeat                   Prints an increasing hearbeat count.
  log-updates start | restart | stop Enable/diable log updates.
  quit                        Exit the command processor
  screensaver                 Unpause all slots which are paused waiting for a
                              screensaver and pause them again on disconnect.
  updates add <id> <rate> <expression> | del <id> | list | clear | reset Enable/disable
                              updates.

Folding@home Client:
  always_on [slot]            Set all or one slot(s) always on.
  bond <ip>:<port> <input> [output] [ip:port] Bond a packet file to a outgoing
                              debug socket connection.
  configured                  Return a PyON message indicating if the client has
                              set a user, team or passkey.
  do-cycle                    Run one client cycle.
  download-core <type> <url>  Download a core.
  finish [slot]               Finish all or one slot(s).
  get-info <category> <key>   Print application information
  info                        Print application information in PyON format
  inject <ip>:<port> <input> [output] [ip:port] Inject a packet file to a
                              listening debug socket. Will wait until packet is
                              processed.
  mask-unit-state             Disable specified unit states.
  num-slots                   Get number of slots in PyON format.
  on_idle [slot]              Set all or one slot(s) on idle.
  option <name> [value]       Get or set a configuration option
  options                     List or set options with their values.
                              If no name arguments are given then all options
                              with non-default values will be listed. If the
                              '-d' argument is given then even defaulted options
                              will be listed. If the '-a' option is given then
                              unset options will also be listed. Otherwise, if
                              option names are provided only those options will
                              be listed.
                              The special name '*' lists all options which have
                              not yet been listed and is affected by the '-d'
                              and '-a' options.
                              If a name argument is followed directly by an
                              equal sign then the rest of the arugment will be
                              used to set the option's value. If instead a name
                              argument is followed immediately by a '!' then the
                              option will be reset to its default value.
                              Options which are set or reset will also be
                              listed.
                              Options are listed as a PyON format dictionary.[-d
                              | -a] | [<name>[! | =<value>]]...
  pause [slot]                Pause all or one slot(s).
  ppd                         Get current total estimated Points Per Day.
  queue-info                  Get work unit queue information in PyON format.
  request-id                  Request an ID from the assignment server.
  request-ws                  Request work server assignment from the assignment
                              server.
  save [file]                 Save the configuration either to the specified
                              file or to the file the configuration was last
                              loaded from.
  shutdown                    Shutdown the application
  simulation-info <slot id>   Get current simulation information.
  slot-add <type> [<name>=<value>]... Add a new slot. Configuration options for
                              the new slot can be provided.
  slot-delete <slot>          Delete a slot. If it is running a unit it will be
                              stopped.
  slot-info                   Get slot information in PyON format.
  slot-modify <id> <type> [<name><! | =<value>>]... Modify an existing slot.
                              Configuration options can be either set or reset
                              using the same syntax used by the 'options'
                              command.
  slot-options <slot> [-d | -a] | [name]... The first argument is the slot ID.
                              See 'options' help for a description of the
                              remaining arguments.
  trajectory <slot id>        Get current protein trajectory.
  unpause [slot]              Unpause all or one slot(s).
  uptime                      Print application uptime
  wait-for-units              Wait for all running units to finish.

Standard Commands:
  add <number> <number>       Add two values
  clear                       Clear the screen
  date [format]               Print the date and time. Optionally, with
                              'format'. See: man strftime
  div <number> <number>       Divide two values
  eq <string> <string>        True if arguments are equal
  eval [expr]...              Evaluate all arguments
  if <cond> <expr1> [expr2]   If 'cond' evaluates to a non-empty string then
                              evalute 'expr1' otherwise, if provided, evaluate
                              'expr2'
  less <string> <string>      True the first argument is lexigraphically less
                              than the second
  mul <number> <number>       Multiply two values
  neq <string> <string>       True if arguments are not equal
  not <expr>                  Invert the truth value of the argument
  sleep <seconds>             Sleep for a number of seconds
  sub <number> <number>       Subtract two values

 

Edited April 12, 2020 by mikkokh

[Jos]

That looks like a list of parameters, not so much the description of a data set.
What about you compile all required information/example of the data and you post it here? 

Jos

[mikkokh]

12 minutes ago, Jos said:

That looks like a list of parameters, not so much the description of a data set.
What about you compile all required information/example of the data and you post it here? 

Jos

 

 

As I just said, it is mentioned several times in the help command feedback in terminal... So, that is not PyON data sample in my last code, that is text output that tells what any terminal command does and if it uses PyON to do so.

[mikkokh]

PyON 1 units
[
  {"id": "01", "state": "RUNNING", "error": "NO_ERROR", "project": 47306, "run": 0, "clone": 2289, "gen": 21, "core": "0x32", "unit": "rsfrd4up3xq7w2n08iol7ynjfizzhqua15", "percentdone": "44.45%", "eta": "4 hours 05 mins", "ppd": "76484", "creditestimate": "13458", "waitingon": "", "nextattempt": "0.00 secs", "timeremaining": "8.07 days", "totalframes": 100, "framesdone": 44, "assigned": "2020-04-12T18:20:24Z", "timeout": "2020-04-13T18:20:24Z", "deadline": "2020-04-20T23:08:23Z", "ws": "139.152.4.231", "cs": "51.214.136.73", "attempts": 0, "slot": "00", "tpf": "4 mins 25 secs", "basecredit": "7462"}
]

Here is a sample of PyON.

[TheXman]

Obviously everything after the first line of your sample is well-formatted JSON.  So if your questions is whether there are better ways to parse JSON values than using a bunch of regular expressions, then the answer is yes.  If you just need to parse out values, then I would suggest using the JSON.au3 UDF that is based on JSMN.  If you need to process a JSON data set (like summing, averaging, counting, or any other type of slicing and dicing of the JSON, then I would suggest using the jq UDF in my signature below.

Below, is a very oversimplified example without any data validation or error checking.  It uses the JSON.au3 UDF.  Neither UDF is included in the standard installation of AutoIt3.  If you plan to use either of them, then you need to download it and place it where your script can include it.

The example below assumes that there may be more than 1 unit in a file.  That is why I use a FOR..IN, instead of just parsing a single object.

Spoiler

#include <Constants.au3>
#include <json.au3> ;<== Change to your UDF's location

example()

Func example()
    Const $JSON_DATA = _
        '[' & _
        '   {"id":"01","state":"RUNNING","error":"NO_ERROR","assigned":"2020-04-12T18:20:24Z","timeout":"2020-04-13T18:20:24Z"},' & _
        '   {"id":"02","state":"PAUSED","error":"NO_ERROR","assigned":"2020-04-13T18:20:24Z","timeout":"2020-04-14T18:20:24Z"}' & _
        ']'

    Local $aoUnits
    Local $i = 0

    ;Decode JSON string into a JSON object (or in this case, an array of JSON objects)
    $aoUnits = Json_Decode($JSON_DATA)
    If @error Then Exit MsgBox($MB_ICONERROR, "ERROR", "Unable to decode json data.")

    ;Display some parsed json values from each unit
    For $oUnit in $aoUnits
        $i += 1
        ConsoleWrite("Unit " & $i & @CRLF)
        ConsoleWrite("id       = " & Json_Get($oUnit, ".id")       & @CRLF)
        ConsoleWrite("state    = " & Json_Get($oUnit, ".state")    & @CRLF)
        ConsoleWrite("assigned = " & Json_Get($oUnit, ".assigned") & @CRLF)
        ConsoleWrite(@CRLF)
    Next
EndFunc

Output:

Unit 1
id       = 01
state    = RUNNING
assigned = 2020-04-12T18:20:24Z

Unit 2
id       = 02
state    = PAUSED
assigned = 2020-04-13T18:20:24Z

 

 

Edited April 13, 2020 by TheXman

[mikkokh]

Thank you.

JSON.au3 UDF seems to be compatible (after removing some text from beginning and end of received console feedback)... However, little more testing is needed to be sure about compatibly.

[TheXman]

You're welcome.

Even if the data doesn't strictly conform to the JSON standard, you still may be able to parse your data since:

Quote

By default jsmn is working in a non-strict mode. It allows you to use it for other useful formats that JSON. In non-strict mode jsmn accepts:

• non-quoted primitive values other than true/false/null/numbers
• only primitive values without a root object

 

The above quote was taken from the original JSMN site.

 

Edited April 14, 2020 by TheXman

[seadoggie01]

I'm not sure that PyON is widely used anymore. I did find this: https://code.google.com/archive/p/pyon/wikis/GettingStarted.wiki

It sounds like it's supposed to be a readable version of python's pickle.