Ideas on Simple Plugin Theory?

[BrendonKoz]

For the past week or so I've been trying to think of a way to allow for a main script to utilize a "plugin"-type architecture (without the use of DLLs). Unfortunately, I am having difficulty in coming up with any type of theoretical solution. I've searched the forums here without much luck (perhaps I was just using the wrong keywords?); the results I found just talked about either using DLL calls, or that some users had figured out their own plugin system but never really went in to any detail as to how they solved it.

My intention is to make the ability of creating additional features to a main program be as easy as possible for anyone looking to build on to my main application. Taking an example from Wordpress, I'm imagining that I would simply search a specific folder for plugin files. Within these files, the plugin would call a function that specified what program "hook" it would attach to, the name of the (internal?) function/method it would have the main program call, its priority level, the auth level allowed to use it, etc...

I've found the AutoIt "Call" and "Execute" methods/functions which I at first thought would be a solution - however the more I thought about it, the less I thought it was a good fit. I can't seem to figure out how to not only load the external scripts, but then load them in to the main script prior to running any non-arbitrary main-program code.

Also, I don't mind if the plugin code needs to be compiled with the core code or not as in this case, the core code will be available to other developers. I was unable to find out whether compiled code could in fact call and run non-compiled AutoIt script code.

If anyone has any ideas, or personal solutions that they've come up with, I would most definitely love to hear them!

Thanks!

[DeltaRocked]

thats an interesting thought. a few weeks / months ago there was topic on these same lines and the guy made a mess out of it ... let me also assist you in this ... will work out a algorithm and will post the same... some time required.

[JohnOne]

"I was unable to find out whether compiled code could in fact call and run non-compiled AutoIt script code."

As far as I am aware, AutiIt3 can run anything which 'can' be ran on a windows machine, using ShellExecute() and/or Run*() functions.

[jvanegmond]

I've done this in a previous project (last worked on 29-Dec-2010). It uses a compiled/uncompiled main script responsible for the plugins, and a compiled script that executes non-compiled plugins on-demand. Using call is not an option, because AutoIt is dynamic except for its functions. You also can't do dynamic includes (you can use a preprocessor that recompiles your application to include the plugins, which is horrible).

Files in the project:

+main.au3 (the main script that can be either compiled or uncompiled)
+wrapper.au3 (only used as source for wrapper.exe)
+wrapper.exe
+\modules\modulename\module.ini (file that contains some information about the plugin - optional)
+\modules\modulename\main.au3

The compiled script that runs the plugins I called wrapper.au3:

#AutoIt3Wrapper_Change2CUI=y

#NoTrayIcon
#include <Constants.au3>

$programLocation = $CmdLine[1]
If StringMid($programLocation, 2, 1) <> ":" Then ; Check for a relative path or absolute path
    ; This is a relative path
    $programLocation = @ScriptDir & "\" & $programLocation ; Make it a absolute path
EndIf
; Get the working dir from the executable, otherwise working dir is current dir and relative paths in module are wrong
$workDir = StringLeft($programLocation, StringInStr($programLocation, "\", 1, -1))

$params = ""
For $i = 2 To $CmdLine[0]
    $params &= '"' & $CmdLine[$i] & '" '
Next

Local $handle = Run(@AutoItExe & ' /ErrorStdOut /AutoIt3ExecuteScript "' & $programLocation & '" ' & $params, @ScriptDir, @SW_HIDE, $STDERR_CHILD + $STDOUT_CHILD)

Local $line
While 1
    $line = StdoutRead($handle)
    If @error Then ExitLoop
    ConsoleWrite($line)
Wend

While 1
    $line = StderrRead($handle)
    If @error Then ExitLoop
    ConsoleWrite($line)
Wend

The main.au3 script loads plugins like so:

expandcollapse popup

Func _ModuleLoadAll()
    Local $iCount = 0
    $search = FileFindFirstFile("modules/*")
    While 1
        $dir = FileFindNextFile($search)
        If @error Then ExitLoop
        $iCount += 1
    WEnd
    FileClose($search)

    Local $modules[$iCount]
    Local $i = 0
    $search = FileFindFirstFile("modules/*")
    While 1
        $dir = FileFindNextFile($search)
        If @error Then ExitLoop

        If Not StringInStr($dir, ".") Then
            $modules[$i] = _ModuleLoad($dir)
        EndIf
        $i += 1
    WEnd
    FileClose($search)
    Return $modules
EndFunc


Func _ModuleLoad($module_dir)
    $config = "modules\" & $module_dir & "\module.ini"
    If Not FileExists($config) Then
        ConsoleWrite("! Failed loading module " & $module_dir & ": " & $config & " not found" & @CRLF)
        Return
    EndIf

    Local $cur_module[5]
    $cur_module[0] = IniRead($config, "Module", "Name", "")
    $cur_module[1] = IniRead($config, "Module", "Description", "")
    $cur_module[2] = @ScriptDir & "\modules\" & $module_dir & "\main.au3"
    $cur_module[3] = IniRead($config, "Module", "Prefix", "")

    Local $commandList[10][2]
    Local $i = 1
    While 1
        $commandListName = IniRead($config, $i, "Name", "")
        $description = IniRead($config, $i, "Description", "")

        If $commandListName = "" Then
            ExitLoop
        EndIf

        $commandList[$i - 1][0] = $commandListName
        $commandList[$i - 1][1] = $description

        $i += 1
    WEnd

    $cur_module[4] = $commandList

    ConsoleWrite("Loaded module " & $cur_module[0] & ". Use prefix: " & $cur_module[3] & @CRLF)

    Return $cur_module
EndFunc

And execution of modules is like:

$exePath = 'wrapper.exe "' & $module[2] & '" ' & $param
$proc = Run($exePath, "", @SW_HIDE, 1 + 2)
While 1
    $read = StdoutRead($proc)
    If @error Then ExitLoop
    If $read Then
        TCPSend($sSocket[$x], $read)
    EndIf
WEnd

An example module that does some file operations called 'file'.

\modules\file\module.ini

[Module]
Prefix=file
Name=File controls
Description=Access and delete local files

[1]
Name=list [path]
Description=Gets a list of directories and files in the [path]

[2]
Name=read [file name]
Description=Reads the first 1000 characters of a file

[3]
Name=rename [original name] [new name]
Description=Renames a file to its new name

[4]
Name=delete [file name]
Description=Deletes a file

modules\file\main.au3

expandcollapse popup

Switch $CmdLine[1]
    Case "read"
        ConsoleWrite("File contents: ")
        $content = FileRead($CmdLine[2])
        ConsoleWrite(StringLeft($content, 1000))
    Case "rename"
        ConsoleWrite("File renamed")
        FileMove($CmdLine[2], $CmdLine[3])
    Case "delete"
        ConsoleWrite("File deleted")
        FileDelete($CmdLine[2])
    Case "list"
        $path = $CmdLine[2]
        If StringRight($path, 1) <> "\" Then
            $path &= "\"
        EndIf
        $path &= "*"
        ; Shows the filenames of all files in the current directory.
        $search = FileFindFirstFile($path)

        ; Check if the search was successful
        If $search = -1 Then
            ConsoleWrite("No files/directories matched the search pattern")
            Exit
        EndIf

        While 1
            $file = FileFindNextFile($search)
            If @error Then ExitLoop

            ConsoleWrite($file & @CRLF)
        WEnd

        ; Close the search handle
        FileClose($search)

    Case Else
        ConsoleWrite("Unknown command: " & $CmdLine[1])
EndSwitch

And as convenience, here is my entire project containing a telnet server where you can modularly add commands: http://dl.dropbox.com/u/10628810/modular%20telnet.rar (this link will die in a few weeks from now, so be quick).

Edit: I was asked if this required AutoIt installed on the computer: It does not. The AutoIt executer gets included in wrapper.exe.

Edit2: Technique used on modular loose coupling:

Edited March 14, 2011 by Manadar

[BrendonKoz]

Sorry for the long time between my initial post and this reply. This isn't work-related, but unfortunately work took over a majority of my life. (Site got hacked early the next morning, cleansed that morning but just finished preventative maintenance.)

Anyhow, thank you very much for the extremely long and in-depth response, as well as offering a downloadable package for testing along with the source pasted here. That's very helpful and much appreciated.

Now, to the questions:

I looked over your response and the associated code without running any of it through AutoIT. I'm curious about what you say here:

AutoIt is dynamic except for its functions. You also can't do dynamic includes (you can use a preprocessor that recompiles your application to include the plugins, which is horrible)

...and what your code does here in _ModuleLoadAll() and _ModuleLoad, as I believe that's similar to what I was thinking of doing (to load the add-ons/modules/plugins) but doesn't that go against what I quote from you above?

Since I'm only on my lunch break at the moment, I tried to quickly open and run the downloaded EXE which unfortunately threw an error:

---------------------------

AutoIt Error

---------------------------

Line 552 (File "C:\Users\<user>\Desktop\modular\wrapper.exe"):

Error: Array variable has incorrect number of subscripts or subscript dimension range exceeded.

---------------------------

OK

---------------------------

I am running Win7 x64, so I thought maybe I'd try running the wrapper.au3 as x86, and then run it as a script (instead of the compiled version), both with the same result:

---------------------------

AutoIt Error

---------------------------

Line 6 (File "C:\Users\<user>\Desktop\modular\wrapper.au3"):

$programLocation = $CmdLine[1]

$programLocation = ^ ERROR

Error: Array variable has incorrect number of subscripts or subscript dimension range exceeded.

---------------------------

OK

---------------------------

At this point I will fully acknowledge and admit that I haven't yet looked at the lines designated in the error message as I'm currently on my lunch break, though I think I'm missing something if the download archive was intended to be a fully operational package; perhaps it wasn't. I'll definitely be looking more closely at this later this evening, however, and hopefully have some time to try some "inspired" functionality out.

[jvanegmond]

It's ok, I'll try to explain a little better what this does so you don't have to invest so much time trying to figure out all the things I am doing.

...and what your code does here in _ModuleLoadAll() and _ModuleLoad, as I believe that's similar to what I was thinking of doing (to load the add-ons/modules/plugins) but doesn't that go against what I quote from you above?

Mostly yes, it does. It was challenging to come up with a system that can dynamically add more functionality to the application. In the end, I found that that is not possible during runtime. All code is loaded when an application starts, so I used an extra application (wrapper.exe) which is started on demand. This has the benefit that you can execute any code as a plugin, but you have to build some interop between the main application and the wrapper application. I did that via startup parameters ($CmdLine) and standard streams (ConsoleWrite and StdOutRead).

wrapper.exe is always started with some command line parameters by main.au3/main.exe. So that you were getting an error running wrapper.exe without providing any command line parameters is not surprising. Try running main.au3, although you will probably need to read the code to get a better understanding.

[BrendonKoz]

Hi again, Manadar. Although I think I understand how to get code to run on-demand now (using interop), I still fail to figure out how your example code works.

You say that wrapper.exe is always started with some command line parameters by main.(au3|exe). However, main.au3 only contains two methods, there's no code in that file that I see which would start running another process, or even one within itself.

In every variation that I've semi-blindly tried to run by passing command line arguments to wrapper.exe (since it seems like that's what takes the arguments), I've gotten no errors now, but also no StdOut responses either.

With the following directory structure, could you provide an example command line statement to run?

-modules/

--download/

---main.au3

---module.ini

--file/

---main.au3

---module.ini

--keyboard/

---main.au3

---module.ini

--media/

---main.au3

---module.ini

--pccontrol/

---main.au3

---module.ini

--run/

---main.au3

---module.ini

-main.exe

-wrapper.exe

?

Sorry if I'm being bothersome. I just feel a bit silly in not being able to figure it out, and I'm hoping if I can see it run, I can then more properly follow the code path.