EasyCodeIt - cross-platform AutoIt implementation

[TheDcoder]

1 hour ago, JockoDundee said:

Do you think, in this case, that its because of variant math that eats up the time?

Yes, that's part of the overhead, but here is a rough high-level view of the over head: floating point math -> execution of statements -> parsing code

The math part goes inside expression evaluation, which is part of executing statements... like the layers of an onion

The machine-code produced by gcc would be very simple, a couple of instructions which just increase a counter, in fact I think most of the overhead would come from initialization of the runtime. Try Tiny C compiler, it may produce a more efficient version.

Also MinGW-w64 provides a native port of GNU GCC to Windows, it is a lot better and it is what I use personally, none of that WSL bull$hit

[JockoDundee]

Just now, TheDcoder said:

it is a lot better and it is what I use personally, none of that WSL bull$hit

I was just trying to make it fairer, I don’t even want to know how bad it would get trounced with the best compiler/env ..

[TheDcoder]

I apologize for the lack of updates, life threw a bunch of stuff at me again so I couldn't focus on the project very much, however I think I can now resume my focus as I have mostly dealt with the other stuff

Behind the scenes I have been working on two helper libraries (UDFs in AutoIt-speak):

1. alloc - This is a wrapper around dynamic memory allocation functions (like malloc), its primary purpose is to keep record of all allocated memory for a certain context.
2. cease - This is a wrapper around setjmp and longjmp, it is similar to the raise_error functions we had in the old parser.

Both of these libraries assist in error recovery, alloc makes it easier to clean up all the memory and cease will make it easier to return to a point in the program where we can safely clean up everything and resume routine operation.

With these two now in place, I can resume work on the parser!

Here is the current code I have if anyone is interested:

Spoiler

alloc.h:

expandcollapse popup

/* 
 * This file is part of EasyCodeIt.
 * 
 * Copyright (C) 2021 TheDcoder <TheDcoder@protonmail.com>
 * 
 * EasyCodeIt is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <https://www.gnu.org/licenses/>.
 */

#ifndef ALLOC_H
#define ALLOC_H

#include <stddef.h>
#include "cease/cease.h"

typedef void *AllocatorFunc(size_t);
typedef void AllocatorFreeFunc(void *);

struct AllocatorNode {
    // FIFO for better performance
    void *ptr;
    struct AllocatorNode *prev;
};

struct Allocator {
    struct AllocatorNode *node;
    AllocatorFunc *alloc;
    AllocatorFreeFunc *free;
    CeasePoint *point;
    char *ctx;
};
typedef struct Allocator Allocator;

Allocator alloc_init(AllocatorFunc *alloc, AllocatorFreeFunc *free, CeasePoint *cease_point, char *def_ctx);
void *alloc_new(Allocator *allocator, size_t size);
void *alloc(Allocator *allocator, size_t size, char *ctx);
void alloc_free(Allocator *allocator, void *ptr);
void alloc_free_all(Allocator *allocator);

#endif

alloc.c:

expandcollapse popup

/* 
 * This file is part of EasyCodeIt.
 * 
 * Copyright (C) 2021 TheDcoder <TheDcoder@protonmail.com>
 * 
 * EasyCodeIt is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <https://www.gnu.org/licenses/>.
 */

#include <stdbool.h>
#include "alloc.h"
#include "cease/cease.h"

Allocator alloc_init(AllocatorFunc *alloc, AllocatorFreeFunc *free, CeasePoint *cease_point, char *def_ctx) {
    return (Allocator){
        .alloc = alloc,
        .free = free,
        .point = cease_point,
        .ctx = def_ctx,
        .node = NULL
    };
}

void *alloc_new(Allocator *allocator, size_t size) {
    AllocatorFunc *aalloc = allocator->alloc;
    AllocatorFreeFunc *afree = allocator->free;
    struct AllocatorNode *node = aalloc(sizeof *node);
    if (!node) goto nomem;
    node->ptr = aalloc(size);
    if (!node->ptr) {
        afree(node);
        goto nomem;
    }
    node->prev = allocator->node;
    allocator->node = node;
    return node->ptr;
    
    nomem: // alloc function returned NULL
    if (allocator->point) cease_mem(allocator->point, allocator->ctx);
    return NULL;
}

void *alloc(Allocator *allocator, size_t size, char *ctx) {
    if (!ctx) return alloc_new(allocator, size);
    char *def_ctx = allocator->ctx;
    allocator->ctx = ctx;
    void *ptr = alloc_new(allocator, size);
    allocator->ctx = def_ctx;
    return ptr;
}

void alloc_free(Allocator *allocator, void *ptr) {
    AllocatorFreeFunc *afree = allocator->free;
    struct AllocatorNode *node = allocator->node;
    struct AllocatorNode *next_node = NULL;
    while (node) {
        if (node->ptr != ptr) {
            next_node = node;
            node = node->prev;
            continue;
        }
        if (next_node) {
            next_node->prev = node->prev;
        } else {
            // node == allocator->node
            allocator->node = node->prev;
        }
        afree(ptr);
        afree(node);
        break;
    }
}

void alloc_free_all(Allocator *allocator) {
    AllocatorFreeFunc *afree = allocator->free;
    struct AllocatorNode *node = allocator->node;
    struct AllocatorNode *prev_node;
    while (node) {
        prev_node = node->prev;
        afree(node->ptr);
        afree(node);
        node = prev_node;
    }
    allocator->node = NULL;
}

cease.h:

expandcollapse popup

/* 
 * This file is part of EasyCodeIt.
 * 
 * Copyright (C) 2021 TheDcoder <TheDcoder@protonmail.com>
 * 
 * EasyCodeIt is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <https://www.gnu.org/licenses/>.
 */

#ifndef CEASE_H
#define CEASE_H

#include <setjmp.h>
#include <stdbool.h>
#include <stdnoreturn.h>

struct CeasePoint {
    jmp_buf jump;
    char *msg;
    bool free_msg;
};

typedef struct CeasePoint CeasePoint;

CeasePoint cease_get_point(void);
noreturn void cease(CeasePoint *point, char *msg, bool free_msg);
noreturn void cease_fmt(CeasePoint *point, char *def, char *fmt, ...);
noreturn void cease_mem(CeasePoint *point, char *context);

#endif

cease.c:

expandcollapse popup

/* 
 * This file is part of EasyCodeIt.
 * 
 * Copyright (C) 2021 TheDcoder <TheDcoder@protonmail.com>
 * 
 * EasyCodeIt is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <https://www.gnu.org/licenses/>.
 */

#define _GNU_SOURCE /* Required to enable (v)asprintf */
#include <setjmp.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdnoreturn.h>
#include "cease.h"

CeasePoint cease_get_point(void) {
    return (CeasePoint){.msg = NULL};
}

noreturn void cease(CeasePoint *point, char *msg, bool free_msg) {
    if (point->free_msg && point->msg) free(point->msg);
    
    point->msg = msg;
    point->free_msg = free_msg;
    
    longjmp(point->jump, true);
}

noreturn void cease_fmt(CeasePoint *point, char *def, char *fmt, ...) {
    va_list args;
    va_start(args, fmt);
    
    char *msg;
    int result = vasprintf(&msg, fmt, args);
    if (result == -1) msg = def;
    
    va_end(args);
    
    cease(point, msg, result != -1);
}

noreturn void cease_mem(CeasePoint *point, char *context) {
    static char *msg = "Failed to allocate memory";
    if (context) {
        cease_fmt(point, msg, "%s when %s", msg, context);
    } else {
        cease(point, msg, false);
    };
}

 

Source: https://forum.dtw.tools/d/28-easycodeit-update-flex-bison-parser/10

[TheSaint]

When I was younger, I used to play ready set jump, and enjoyed the high jump as well as the long jump ... the short jump not so much or the low jump.  

Very good bud, you continue to amaze me with your work. Now highjmp to it. 

[TheDcoder]

Quote

Hi everyone,

It's been a little over 2 months since the last update, I have been working on and off since then. Finally I have something that I can share with you guys

I just completed writing the first iteration of the parser! It should be able to parse all kinds of expressions, including array/object access and function calls

You can download the latest build from here: https://github.com/TheDcoder/EasyCodeIt-WIP/releases/tag/test-2

It's a 32-bit build (special thanks to TheSaint for performing a pre-release test on his old 32-bit laptop) so it should run on all hardware.

Usage Instructions:

1. Extract contents of the archive into a folder
2. Use cmd or a shell of your choice to run eci (there's no GUI yet)
3. Write any expression in a text file to parse
4. Call eci.exe with the path to the said text file
5. Examine the JSON output

Here is an example from my computer:

J:\Projects\EasyCodeIt\build_win>type ..\build\example_scripts\eci.txt
-(($foo + 2) / 3) * 4

J:\Projects\EasyCodeIt\build_win>eci ..\build\example_scripts\eci.txt
{
    "op": "Multiply",
    "args": [
        {
            "op": "Invert",
            "args": [
                {
                    "op": "Divide",
                    "args": [
                        {
                            "op": "Add",
                            "args": [
                                {
                                    "ident": "$foo"
                                },
                                2.0
                            ]
                        },
                        3.0
                    ]
                }
            ]
        },
        4.0
    ]
}

Please test it and report any bugs or crashes!

For those who are interested in the source code, they can browse the full tree here, the juicy bits are inside the parser directory, especially the parser.y file in it!

Source: https://forum.dtw.tools/d/29-easycodeit-expression-parser-download-available

You can post your test results here, but I encourage you to post it in the dedicated forum topic if possible, thank you!

Edited November 1, 2021 by TheDcoder

[TheDcoder]

I already found a bug with function calls, I have pushed a small update to fix that, please download the new version from the updated link.

Here is the monster expression which was used to find the bug:

expandcollapse popup

> cat example_scripts/expression.au3
MsgBox($MB_INFO, "Math!", -(($foo + 2) / 3) * 4)[666].fortyTwo)
> ./eci example_scripts/expression.au3
{
    "op": "Access",
    "args": [
        {
            "op": "Access",
            "args": [
                {
                    "op": "Call",
                    "args": [
                        {
                            "ident": "MsgBox"
                        },
                        [
                            {
                                "op": "No Operation",
                                "args": [
                                    {
                                        "ident": "$MB_INFO"
                                    }
                                ]
                            },
                            [
                                {
                                    "op": "No Operation",
                                    "args": [
                                        "\"Math!\""
                                    ]
                                },
                                [
                                    {
                                        "op": "Multiply",
                                        "args": [
                                            {
                                                "op": "Invert",
                                                "args": [
                                                    {
                                                        "op": "Divide",
                                                        "args": [
                                                            {
                                                                "op": "Add",
                                                                "args": [
                                                                    {
                                                                        "ident": "$foo"
                                                                    },
                                                                    2.0
                                                                ]
                                                            },
                                                            3.0
                                                        ]
                                                    }
                                                ]
                                            },
                                            4.0
                                        ]
                                    },
                                    null
                                ]
                            ]
                        ]
                    ]
                },
                666.0
            ]
        },
        {
            "ident": "fortyTwo"
        }
    ]
}

 

[JockoDundee]

But what say your parser to the controversial, yet simple expression:

-3^2

I would try it myself, but I’m driving

[TheDcoder]

@JockoDundee It's favoring the "more logical" approach of inversion before exponentiation

> ./eci /tmp/controversy.txt
{
    "op": "Raise",
    "args": [
        {
            "op": "Invert",
            "args": [
                3.0
            ]
        },
        2.0
    ]
}

Here is the precedence used by the parser (starting from lowest to highest):

/* Operators */
%precedence '?'
%precedence ':'
%left AND "And" OR "Or"
%left LT '<' GT '>' LTE "<=" GTE ">=" EQU '=' NEQ "<>" SEQU "=="
%left '&'
%left '+' '-'
%left '*' '/'
%left '^'
%left NOT "Not"
%precedence INVERSION
%precedence '.'
 /* WORKAROUND: Bison can't handle "sandwhich" operators which surround the 2nd part of a binary expression */
%precedence '['
%precedence '('
%precedence GROUPING

 

[TheSaint]

On 11/1/2021 at 11:13 AM, TheDcoder said:

Here is the monster expression which was used to find the bug

Just too monstrous for me to appreciate the finer workings of what parses through your mind. 

[TheDcoder]

@TheSaint It's meaningless, like all monsters

Useful for testing all kinds of edge cases though. Some developers even use something called "fuzz testing" where they supply totally random input to a program to see if it crashes

[TheSaint]

9 hours ago, TheDcoder said:

Some developers even use something called "fuzz testing" where they supply totally random input to a program

Is that similar to regular use of a Windows PC? 

i.e. the random event of the Ghost In The Machine. 

P.S. I have always suspected some coding nerds of doing that instead of Easter Eggs ... or maybe being super bright like they are they think that's what Easter Eggs are. 

[BakedCakes]

A couple of questions:

1. Why create an entirely new language instead of making EasyCodeIt be a Lua or Python library/module? Making the language AutoIt-like would allow for easier translation from one to another indeed, but using an existing language outweighs any advantage this brings, as that would allow one to use many-many existing Lua and Python libraries in their EasyCodeIt project, allow to use existing tools: debugging, code analysis, build, testing, etc. Also, a knowledge of a popular programming language is quite useful, when learning AutoIt-like language would be mostly a dead end as nothing else uses it -- there are barely any job posts seeking AutoIt developers, while many seek Python and (less so) Lua developers.
2. How are you going to provide a similar to AutoIt functionality on Linux without Linux having a single unified way of doing things, unlike Windows? Just sending keystrokes to a window or controlling a window differs so much depending on whether you are using: Xorg, Wayland, etc. and some operations might be supported only on Xorg but not on Wayland, and the other way around. This seems to get messy rather quickly, as things working on one Linux system might not work on another Linux system.

 

[TheDcoder]

Hi @BakedCakes,

Thank you for showing interest, however I think you've misunderstand the goal of this project, perhaps you might want to read this thread which is where the idea started:

TL;DR EasyCodeIt is not an GUI automation-centric language like AutoIt, it's more about providing AutoIt's syntax coupled with a featureful standard library just like Python.

[BakedCakes]

Ah, I see, looks like I misunderstood the goal of the project.

[TheDcoder]

Quote

Hello everyone,

I know that it's been a while since the last update... 1 year 4 months 21 days to be exact. A passionate member of the AutoIt forum Chris ("JockoDundee") also passed away shortly after the previous update. So this update is dedicated to him

A lot of stuff happened in-between and this project essentially went into a coma given my lack of motivation to keep it going forward. I myself was going through a depressive period where I was very unproductive, I even stopped working on work-related projects, only working on existing client contracts occasionally to keep the boat afloat essentially, relying on savings to cover the gap.

Anyway, keeping all of that aside. Today I finally convinced myself to get back to what I was working on behind the curtain.

I had already begun work on parsing statements, but hit a road-block while trying to create a list of expressions separated by new lines (statements, essentially).

Today I got the idea to ask ChatGPT, the AI, for help after reading countless articles and posts on it. Apparently it's even on track to replace programmers entirely... at-least that's what some people say, I have my reservations

I have to admit though, while not a complete replacement, it is a very competent wing-man and it actually helped me fix a bug which I had banging my head around trying to debug! Of course it wasn't perfect, but you can easily guide it to the right direction if you know what you are doing.

Here's what the issue I had to deal with, I created a new `expression_list` rule to match a list of expressions:

expression_list:
	  expression {$$ = exprlist_from_expr(&$1, NULL);}
	| expression '\n' expression_list {$$ = exprlist_from_expr(&$1, &$3);} */
	| expression_list ',' expression {$$ = exprlist_from_expr(&$3, &$1);}
	| expression_list '\n' expression {$$ = exprlist_from_expr(&$3, &$1);}

But it kept throwing a syntax error after parsing the first expression... and I could never figure out why. ChatGPT came to my rescue, and after making it familiar with the code and issue, it suggested the following approach:

expression_list:
	  expression {$$ = exprlist_from_expr(&$1, NULL);}
	| expression '\n' expression_list {$$ = exprlist_from_expr(&$1, &$3);} */
	| expression_list ',' expression {$$ = exprlist_from_expr(&$3, &$1);}
	| expression_list NL expression {$$ = exprlist_from_expr(&$3, &$1);}
	| expression_list NL {$$ = $1;} // Allow multiple new lines between expressions

And this actually works like a charm! The change is simple, the literal `'\n'` token is replaced with the `NL` token from the lexer.

To be honest I still don't know why this worked and not the older code, but I'm glad to be past this road-block. Maybe it's because the lexer ate up the `\n` character so bison couldn't see it?

I plan to continue work on this so we have a working parser, and I can finally begin work on the interpreter and runtime

Source: https://forum.dtw.tools/d/28-easycodeit-progress-flex-bison-parser/15