EasyCodeIt - cross-platform AutoIt implementation

[TheDcoder]

EasyCodeIt

A cross-platform implementation of AutoIt

Introduction:

EasyCodeIt is an attempt at implementing a programming language which is backward compatible with a sub-set of AutoIt which would work across multiple platforms with many aspects being agnostic to the platform-specific differences.

Currently the primarily targeted platforms are Linux and Windows, more platforms may be supported in the future as per popular demand. For example it should be easy enough to port to Unix-like platforms such as BSD and Mac with the help of maintainers who are familiar with them.

The main motivation behind the creation of this project is the lack of a proper easy-to-use scripting language in Linux, while there are numerous scripting languages which natively support it, none of them are as user-friendly and easy to use as AutoIt. (The "Easy" in "EasyCodeIt" reflects this)

There was a previous thread in which the project was originally started, but it got too big and the discussion is too cluttered and hard to follow for any new readers, here is the thread for those who are interested in reading it:

Progress:

• Frontend
  • ✅ Tokenizer
  • 🚧 Parser (work in progress)
    • ✅ Expressions
    • 👷‍♂️ Statements (current priority)
• Backend
  • ⏰ Interpreter (scheduled)

I am currently working on expression parsing and syntax tree building.
--
This section will be updated in the future with new progress. To stay notified 🔔 follow this thread to get update notifications (by using the "Follow" button on the top-right corner of this page) and 👁️ watch the GitHub repository to get any new activity on your feed.

Code:

The code is available on GitHub, please 🌟 star the project if you like it and would like to show your support, it motivates me a lot! (Also don't forget to 👍 Like this post )

Chat:

I created a room in Matrix (an open-source federated chat system) for EasyCodeIt, you can join it to chat with me and others who are in the room, it might be a good way to get the latest status updates from me

You can preview the room without joining, and you don't need to be connected to it 24/7 like IRC. It works a bit like Slack for those who are familiar with it.

Forum:

I have created a dedicated forum to post more frequent updates, possibly in their own threads to better organize the discussion. Please sign-up and follow the blog section to get updates. By the way, you can also post pretty much anything there, both technical and non-technical stuff, it is intended to be a hangout for techies but not only for them. So casual discussions, funny cat videos etc. are allowed!

Edited February 22, 2021 by TheDcoder
Add Forum section and update progress

[TheDcoder]

Reserved for future use

[JLogan3o13]

Just curious why a second topic, on the very same thread, in the very same place on the forum?

[TheDcoder]

@JLogan3o13 This is supposed to the the official topic of the project, the other thread was just a discussion that I started before I planned to start work on it. And as I already mentioned, the other thread got too big and it is not very easy to get into for new readers.

So I hope to keep this thread cleaner and also offer a good overview in the first posts of this thread

[JockoDundee]

Question:

Are there any intellectual property issues that could arise, vis a vis, Autoit?

Clearly, though Autoit is free, it’s not AFAIK open-source.  
OTOH, you’re not copying any source code, so there’s that.

But is the specification itself possibly something that can be protected?

Not trying to cause you trouble, just seems interesting...

[TheDcoder]

@JockoDundee That's a good question! I am not a lawyer obviously but I will write my thoughts about it here.

As you have mentioned, I am not using any part of the source or binaries from AutoIt. But what I am doing is making a product which is similar to it... but AutoIt itself also shares similarities with other BASIC language, not to mention that it was open-sourced under GPL at one point, so there is that

Even if I am kind of in a gray area, I am acting in good faith and not ripping-off AutoIt for a quick buck, so I hope Jon and rest of the AutoIt team don't have any issue with my endeavor

[JockoDundee]

17 minutes ago, TheDcoder said:

but AutoIt itself also shares similarities with other BASIC language

IP Lawyer on Cross-Examination: So to be clear, Mr. TheDcoder, you claim that your award winning  “EasyCodeIt” product is your own work, fashioned in the style of the BASIC family of languages perhaps, but distinctive nonetheless?

TheDcoder: Yes.

IP Lawyer: In that case, I’m sure you wouldn’t mind explaining a few of your design decisions.  Can you tell the court after you straightforwardly created functions such as MouseMove(), MouseClick() and MouseGetPos() , what did you call the function to determine when the mouse has been clicked?

TheDcoder:  If you’re referring to _IsPressed(“01”), I can explain; you see the underscore means UDF, and if you know hex then...

IP Lawyer: A likely story.  Why don’t you explain why in the thirty plus Array functions, you have no _ArrayCompare() routine?

TheDcoder: Actually, I was just working on that...

IP Lawyer: Sure you were, pal. Ok, answer me this: Why did you make the WinSetState function work like this WinSetState($Hwnd, “”, @SW_SHOW) while reversing the arguments in the case of GuiSetState(@SW_SHOW, $Hwnd)?

TheDcoder: I like to mix it up?

IP Lawyer: Final question, what is the purpose of Dim?

TheDcoder: Nothing says BASIC like Dim!

Judge: Case dismissed!

[TheDcoder]

LOL

By the way, I am not actually going to implement the Automation related functions, I want to create a general-purpose language which can be expanded by the user according to their needs (so there could be an UDF with those functions )

Should probably mention this in the first post

[TheDcoder]

I am still working on the expression parser, the code is not complete but since it has almost been a week since the last update, so I thought I'd give you guys a sneak peak

Here is a screenshot with the graphical representation of an expression's parse tree:

The expression is pretty simple: 1 - 2 / 3 * 4

But the final result looks complex and huge because there are multiple layers of nodes for each bit of info... all thanks to C's strict type system, where you can't put anything anywhere like you usually would in a dynamically typed language like AutoIt.

Here is a snippet of the code (the expression parsing function) for those are interested:

Spoiler

struct Expression expression_get(struct Token *tokens, size_t count) {
	static char *err_mem_ctx = "parsing expression";
	struct Expression expression;
	
	int target_precedence = -1;
	enum Operation target_operation = OP_NOP;
	size_t target_position;
	
	// Check if there is only 1 non-whitespace token
	bool single_token = false;
	for (size_t i = 0; i < count; ++i) {
		if (tokens[i].type == TOK_WHITESPACE) continue;
		if (single_token) {
			single_token = false;
			break;
		}
		single_token = true;
		target_position = i;
	}
	
	if (single_token) {
		struct Token *token = tokens + target_position;
		expression.op = OP_NOP;
		
		struct Operand *term = malloc(sizeof(struct Operand));
		if (!term) raise_mem(err_mem_ctx);
		term->type = OPE_PRIMITIVE; // TODO: Support other types
		
		struct Primitive *value = malloc(sizeof(struct Primitive));
		if (!value) raise_mem(err_mem_ctx);
		switch (token->type) {
			case TOK_NUMBER:
				value->type = PRI_NUMBER;
				value->number = 1337; // FIXME: Parse numbers
				break;
			case TOK_STRING:
				value->type = PRI_STRING;
				value->string = malloc(token->data_len + 1);
				strncpy(value->string, token->data, token->data_len);
				break;
			case TOK_BOOL:
				// FIXME: Parse booleans
				break;
		}
		term->value = value;
		
		expression.operands = term;
	} else {
		for (size_t i = 0; i < count; ++i) {
			if (tokens[i].type != TOK_OPERATOR) continue;
			if (tokens[i].op_info.precedence > target_precedence) {
				target_precedence = tokens[i].op_info.precedence;
				target_operation = tokens[i].op_info.op;
				target_position = i;
			}
		}
		
		expression.op = target_operation;
		if (expression.op) {
			expression.operands = malloc(sizeof(struct Operand) * 2);
			if (!expression.operands) raise_mem(err_mem_ctx);
			expression.operands[0].type = OPE_EXPRESSION;
			expression.operands[1].type = OPE_EXPRESSION;
			expression.operands[0].expression = malloc(sizeof(struct Expression));
			if (!expression.operands[0].expression) raise_mem(err_mem_ctx);
			expression.operands[1].expression = malloc(sizeof(struct Expression));
			if (!expression.operands[1].expression) raise_mem(err_mem_ctx);
			*expression.operands[0].expression = expression_get(tokens, target_position);
			*expression.operands[1].expression = expression_get(tokens + target_position + 1, count - target_position - 1);
		}
	}
	
	return expression;
}

 

There are no doubt many bugs in there, and a lot of the stuff needs to be completed, but I will try to post weekly updates to show the progress.

[JockoDundee]

Strict-typing is possibly your friend here, less weirdness, though more code, no?

On 11/30/2020 at 3:38 PM, TheDcoder said:

But the final result looks complex and huge because there are multiple layers of nodes for each bit of info... all thanks to C's strict type system,

Can you do it without recursion though?

[TheDcoder]

11 hours ago, JockoDundee said:

Strict-typing is possibly your friend here, less weirdness, though more code, no?

Indeed, that's one of the advantages of strict-typing, less ambiguity and surprises in the future

11 hours ago, JockoDundee said:

Can you do it without recursion though?

I am not sure, I can think of a way to do this iteratively using goto but not sure if it would be enough to make it work. In any case, a recursive function is simpler in this case, so we can look into non-recursive approaches in the future if needed.

[JockoDundee]

8 hours ago, TheDcoder said:

I am not sure, I can think of a way to do this iteratively using goto but not sure if it would be enough to make it work.

My bad, I thought it wasn’t using recursion when I asked.  I looked but missed the final lines of code that you published.

*expression.operands[0].expression = expression_get(tokens, target_position);
*expression.operands[1].expression = expression_get(tokens + target_position + 1, 

so here it almost seems like the struct itself is recursively defined, though I know that’s not the case.
Are there just 2 elements to the operands array?

What is the difference between the tokens struct and the expressions struct?

[TheDcoder]

@JockoDundee Ah I see. You are right that the struct isn't recursively defined (that's impossible in C), what I do is have a pointer for the same struct

24 minutes ago, JockoDundee said:

Are there just 2 elements to the operands array?

Technically it is not an array, it is just a pointer to an operand structure, but thanks to pointer arithmetic in C, we can treat it like an array if we allocate enough space to contain multiple structures. The number of elements is supposed to derived from the operation, so for example with OP_ADD (which is binary addition: 1 + 2) the code would assume 2 elements.

There can be any number of operands (or arguments) for an operation, but we only use a maximum of 3 (the conditional `?:` operator takes 3 arguments, which is also why it is called the ternary operator).

24 minutes ago, JockoDundee said:

What is the difference between the tokens struct and the expressions struct?

A "Token" represents an individual unit/value in the code, while an "Expression" represents a set of operations that are done on these tokens. In simple English, tokens are the numbers and symbols while the expression is the whole equation

Edited December 3, 2020 by TheDcoder
Add link to pointer arithmetic

[rcmaehl]

All I want is unsigned 64bit bitwise operations

[TheDcoder]

@rcmaehl Maybe in the future but you will have to settle with the Bitwise functions for now!

[Nine]

9 hours ago, rcmaehl said:

All I want is unsigned 64bit bitwise operations

Look in my signature for it...

[rcmaehl]

1 hour ago, Nine said:

Look in my signature for it...

Trying not to have to FileInstall dlls currently. I don't need high speed, the function is rarely called, but when it does I need it to be able to accurately represent a bitmask. Currently trying to manually code it myself but running into an issue where there's some loose typing shenanigans and the math breaks.

The string value is right but not the variable value

Edited December 7, 2020 by rcmaehl

[TheDcoder]

Doesn't AutoIt have 64-bit integer support? In any case, I will make sure to add this in EasyCodeIt, perhaps as an optional feature

[Nine]

3 minutes ago, TheDcoder said:

Doesn't AutoIt have 64-bit integer support?

AutoIt only supports 32bits bitwise operations (see remark in BitAND)

[Nine]

On 12/7/2020 at 12:27 PM, rcmaehl said:

Trying not to have to FileInstall dlls currently. I don't need high speed, the function is rarely called, but when it does I need it to be able to accurately represent a bitmask.

Then use my slow script :

Spoiler

expandcollapse popup

Func _BitAND64($iValue1, $iValue2)
  If VarGetType($iValue1) <> "Int64" And VarGetType($iValue2) <> "Int64" Then Return BitAND($iValue1, $iValue2)
  $iValue1 = __DecToBin64($iValue1)
  $iValue2 = __DecToBin64($iValue2)
  Local $aValueANDed[64]
  For $i = 0 To 63
    $aValueANDed[$i] = ($iValue1[$i] And $iValue2[$i]) ? 1 : 0
  Next
  Return __BinToDec64($aValueANDed)
EndFunc   ;==>_BitAND64

Func _BitOR64($iValue1, $iValue2)
  If VarGetType($iValue1) <> "Int64" And VarGetType($iValue2) <> "Int64" Then Return BitOR($iValue1, $iValue2)
  $iValue1 = __DecToBin64($iValue1)
  $iValue2 = __DecToBin64($iValue2)
  Local $aValueORed[64]
  For $i = 0 To 63
    $aValueORed[$i] = ($iValue1[$i] Or $iValue2[$i]) ? 1 : 0
  Next
  Return __BinToDec64($aValueORed)
EndFunc   ;==>_BitOR64

Func _BitXOR64($iValue1, $iValue2)
  If VarGetType($iValue1) <> "Int64" And VarGetType($iValue2) <> "Int64" Then Return BitXOR($iValue1, $iValue2)
  $iValue1 = __DecToBin64($iValue1)
  $iValue2 = __DecToBin64($iValue2)
  Local $aValueXORed[64]
  For $i = 0 To 63
    $aValueXORed[$i] = (($iValue1[$i] And (Not $iValue2[$i])) Or ((Not $iValue1[$i]) And $iValue2[$i])) ? 1 : 0
  Next
  Return __BinToDec64($aValueXORed)
EndFunc   ;==>_BitXOR64

Func _BitNOT64($iValue)
  If Not VarGetType($iValue) = "Int64" Then Return BitNOT($iValue)
  $iValue = __DecToBin64($iValue)
  For $i = 0 To 63
    $iValue[$i] = Not $iValue[$i]
  Next
  Return __BinToDec64($iValue)
EndFunc   ;==>_BitNOT64

Func _BitRotate64($iValue, $iShift)
  If VarGetType($iValue) <> "Int64" Then Return BitRotate($iValue, $iShift, "D")
  $iValue = __DecToBin64($iValue)
  Local $iTmp
  If $iShift < 0 Then ; rotate right
    For $i = 1 To Abs($iShift)
      $iTmp = $iValue[0]
      For $j = 1 To 63
        $iValue[$j - 1] = $iValue[$j]
      Next
      $iValue[63] = $iTmp
    Next
  Else
    For $i = 1 To $iShift
      $iTmp = $iValue[63]
      For $j = 63 To 1 Step -1
        $iValue[$j] = $iValue[$j - 1]
      Next
      $iValue[0] = $iTmp
    Next
  EndIf
  Return __BinToDec64($iValue)
EndFunc   ;==>_BitRotate64

Func _BitShift64($iValue, $iShift, $bSigned = True)
  If $iShift <= 0 Then $bSigned = True
  If VarGetType($iValue) <> "Int64" Then
    If $bSigned Then Return BitShift($iValue, $iShift)
    If $iShift > 0 Then
      $iValue = BitAND(BitShift($iValue, 1), 0x7FFFFFFF)
      Return BitShift($iValue, $iShift-1)
    EndIf
  EndIf
  $iValue = __DecToBin64($iValue)
  Local $iTmp
  If $iShift > 0 Then ; shift right
    $iTmp = $bSigned ? $iValue[63] : 0
    For $i = 1 To Abs($iShift)
      For $j = 1 To 63
        $iValue[$j - 1] = $iValue[$j]
      Next
      $iValue[63] = $iTmp
    Next
  Else
    For $i = 1 To Abs($iShift)
      For $j = 63 To 1 Step -1
        $iValue[$j] = $iValue[$j - 1]
      Next
      $iValue[0] = 0
    Next
  EndIf
  Return __BinToDec64($iValue)
EndFunc   ;==>_BitShift64

Func __DecToBin64($iDec)
  Local $tI64 = DllStructCreate("int64 num"), $aBin[64], $iVal
  Local $tI32 = DllStructCreate("align 4;uint low;uint high", DllStructGetPtr($tI64))
  $tI64.num = $iDec
  For $i = 0 To 31
    $iVal = 2 ^ $i
    $aBin[$i] = BitAND($tI32.low, $iVal) ? 1 : 0
    $aBin[$i + 32] = BitAND($tI32.high, $iVal) ? 1 : 0
  Next
  Return $aBin
EndFunc   ;==>__DecToBin64

Func __BinToDec64($aBin)
  Local $tI32 = DllStructCreate("align 4;uint low;uint high"), $iVal
  Local $tI64 = DllStructCreate("UINT64 num", DllStructGetPtr($tI32))
  For $i = 0 To 31
    $iVal = 2 ^ $i
    $tI32.low += $aBin[$i] ? $iVal : 0
    $tI32.high += $aBin[$i + 32] ? $iVal : 0
  Next
  Return $tI64.num
EndFunc   ;==>__BinToDec64

 

 

Edited December 9, 2020 by Nine
corrected small bugs and streamlined code