TCP and Encryption included or not?

[layer]

Hey guys, 2 questions:

1.) Are we using the encryption things JS made?

2.) Why is Jon thinking of not including the TCP funcs in the official release?

Thanks!

EDIT: Could the mods/admins delete one of these threads, sorry for the double post!

Edited July 12, 2005 by layer

[FuryCell]

2.) Why is Jon thinking of not including the TCP funcs in the official release?

<{POST_SNAPBACK}>

What??? I hope this does not happen. TCP opens up so many possibilites for AutoIt. Edited July 13, 2005 by SolidSnake

[Valik]

Virtually every function in AutoIt can be re-implemented via DllCall/DllStruct* (I can't think of one off the top of my head that can't aside from DllCall/DllStruct* or those that operate directly on AutoIt's underlying data structure). In other words, stop panicing; AutoIt-only (No DLL) TCP support is here forever, whether that is built-in or via a library written in "pure" AutoIt.

[layer]

NVM, re-read Valik's post, now I understand

Thanks!

Edited July 13, 2005 by layer

[Jon]

I was umming and ahhing over the TCP functions as I just don't see a "general" use for them. Ping, InetGet, et all are all functions that are accessible and useful to a large portion of the community. TCP stuff on the other hand, I can't even imagine ever needing to use them. So I'm thinking a dll is more appropriate. Even if it is a dll that we supply as part of the install.

The exe size has rocketed from 111KB to 150+ so just don't want any code in there that is ultra niche

[/dev/null]

needing to use them.  So I'm thinking a dll is more appropriate.  Even if it is a dll that we supply as part of the install.

a separate dll is not bad, but then you'll have to add that dll to the compiled EXE as well, otherwise it would be a "pain" to create a single distributable EXE. O.K. one

could use FileInstall(), but that's not quite the same...

BTW: It would be interesting to know the planned future of AutoIT. Will it become a general purpose scripting language like perl or python (EDIT: while much of the functionality is already available in AuotIT), or will it become more "GUI" centric, having more commands to "manipulate" windows programs. That general decision will most probably have an influence on the question if AutoIT needs built-in networking support or not.

Cheers

Kurt

Edited July 13, 2005 by /dev/null

[layer]

Oh well, I guess it's up to Jon to decide, looks like he's made his choice already

[FuryCell]

In other words, stop panicing; AutoIt-only (No DLL) TCP support is here forever, whether that is built-in or via a library written in "pure" AutoIt.

<{POST_SNAPBACK}>

Thanks. I thought we were going to lose TCP for a moment. The libary of some sort sounds like a good idea.

P.S. I have a question about plugins. When and if they are implemented will they have to be fileinstalled or will they just be included somehow like #Include.

Edited July 13, 2005 by SolidSnake

[Angel]

I was umming and ahhing over the TCP functions as I just don't see a "general" use for them.  Ping, InetGet, et all are all functions that are accessible and useful to a large portion of the community.  TCP stuff on the other hand, I can't even imagine ever needing to use them.  So I'm thinking a dll is more appropriate.  Even if it is a dll that we supply as part of the install.

The exe size has rocketed from 111KB to 150+ so just don't want any code in there that is ultra niche

<{POST_SNAPBACK}>

Jon, I bet that this feature creep will continue to be a problem for the foreseeable future. Thus, perhaps a long term solution should be found. Today you have to choose whether to leave out TCP, tomorrow it will be something else...

I guess that this has perhaps been brought up earlier by someone else, but do you think that it could be possible to have the compiler somehow "split" the autoit executable and only include in the EXE the parts that are necessary for the script?

For instance, if no GUI functions are used, the GUI part could be left out. Same thing for TCP, etc. This would solve the problem that many people have (me included) with having to "manualy" distribute DLLs with our EXEs, as one of the beauties of AutoIt is that you can make small but powerful _stand-alone_ programs.

I don't mean that the compiler should be super-smart and completelly optimize the size of the executable. It would be good enough if it could be split into 2 or 3 core but optional functionalities, like GUI, TCP, etc (using some partitioning that makes sense size-wise).

I understand that this could be hard to do, specially with functions like Execute, Eval, etc. However perhaps the compiler could also let you force to add some functionality even if the compiler thought that it was not needed. By default the complier would add the whole functionality if Eval, Execute, etc were used, and the user could chose to leave out some parts by using the Compiler GUI.

For newbies this would be transparent (other than not so optimized EXE files if they use Eval, etc) and for more experienced users it would be great (I think!).

Sounds like a fun project, doesn't it?

What do yo guys think?

Angel

[Valik]

An optimizing compiler like that is going to have to have multiple AutoItSC.bin files specially compiled with some features missing. It'll then also have to have a table of what functions can be found in what AutoItSC.bin. Depending on the granularity of how things should be "separated", this will astronomically increase the size of the AutoIt package with a very small decrease in the size of a compiled executable. I personally don't want to waste my hard drive space having 98298592 copies of variously compiled AutoItSC.bin files (Remember that AutoItSC.bin is not UPX'ed, either, so the file size is roughly double per file).

Also, keep in mind, even if AutoIt swells up to be 700 KB per single compiled binary, that is:

• Less than the average size of most normal MFC applications (Based on my observations).
• Less than the size of the Visual Basic run-times.
• Far less than the size of the .NET run-time which is necessary for C#, VB .NET (VB6 is no longer officially supported, by-the-way, so its .NET or no support), or managed C++.
• Small enough to still fit on a 3.5 inch disk.
• Will still be 100% stand-alone requiring only Windows system DLL's to operate (Unless you as the user have specifically made your own requirement on something else).

So even if AutoIt becomes "huge" based on looking at the size of a single compiled script, it still absolutely crushes other popular development languages in terms of requirements. Not a single one of those languages (Except MFC... statically linked) support stand-alone executables. AutoIt does.

[FuryCell]

Jon, I bet that this feature creep will continue to be a problem for the foreseeable future. Thus, perhaps a long term solution should be found. Today you have to choose whether to leave out TCP, tomorrow it will be something else...

I guess that this has perhaps been brought up earlier by someone else, but do you think that it could be possible to have the compiler somehow "split" the autoit executable and only include in the EXE the parts that are necessary for the script?

For instance, if no GUI functions are used, the GUI part could be left out. Same thing for TCP, etc. This would solve the problem that many people have (me included) with having to "manualy" distribute DLLs with our EXEs, as one of the beauties of AutoIt is that you can make small but powerful _stand-alone_ programs.

I don't mean that the compiler should be super-smart and completelly optimize the size of the executable. It would be good enough if it could be split into 2 or 3 core but optional functionalities, like GUI, TCP, etc (using some partitioning that makes sense size-wise).

I understand that this could be hard to do, specially with functions like Execute, Eval, etc. However perhaps the compiler could also let you force to add some functionality even if the compiler thought that it was not needed. By default the complier would add the whole functionality if Eval, Execute, etc were used, and the user could chose to leave out some parts by using the Compiler GUI.

For newbies this would be transparent (other than not so optimized EXE files if they use Eval, etc) and for more experienced users it would be great (I think!).

Sounds like a fun project, doesn't it? 

What do yo guys think?

Angel

<{POST_SNAPBACK}>

Funny. I was thinking about this last night becuase I remembered hearing somthing about some smart Quick Basic Compiler because I was working on a really old computer (made in 1991. ).

I like this idea but even though I am not in the developer group it sounds complicated and like it might have to wait a while (maybe V4.)

[Nutster]

The way I see this is not having several versions of AutoItSC.bin, but several libraries that get linked based on what is needed. For example, AutoItSC.bin would have the core functionality, AutoItGUI.bin would have the functionality for GUI creation functions, AutoItTCP.bin would have the TCP functions, AutoItRegExp.bin would have the Regular Express stuff (it is not that small either), AutoItParse.bin would have the have the parsing code, etc. Only those parts that are needed based on what functions are called in the script would be linked into the executable. If Execute() is used, all bets are out the window and everything will need to be included, just in case somebody calls pretty much anything, but otherwise we could leave out the parts that are not needed, without having lots of sections.

Hmm, how to do this? Have a dispatch table that looks into the various libraries. If a library is not called, just drop it. But each library will need to be dynamically linked in order to do this. Nuts, we would need to have all the .bin's above be .dll instead, that automagically install at the start of run (first checking if the DLL's already exist), which could slow things down a lot. Hmm, What are we doing for plug-ins?

[Valik]

Jon'll never go for that, David. I can write you a 100% seamless plug-in architecture but Jon doesn't like the DLL part of the idea. You could also use COM to achieve the same thing, although that is a bit more work and definitely slower with the overhead of COM. My implementation (concepts) were similar in ways to COM (Although I did it before I learned about COM). There were some design flaws due to my lack of knowledge of compilers and C++, but I did make at least one working prototype, so I know the concept is at least doable.

However, doing it requires massive rewriting of AutoIt. At best, it would be as complex as re-writing each suite of related functions as a class derived from the plug-in-architecture base. At worst, there would be functions that relied on other functions that aren't directly related from the outside but internally are. This would lead to broad and often wrong categorization or duplicated code (Both bad in their own right). Another thing that would need added would be a hook system so that background things that need done such as window polling could be done during the times AutoIt is "idle".

Then there is the problem with DLL's themselves. They will all duplicate some C-runtime code, even if they are dynamically linked against msvcrt.dll. I doubt any of the component DLL's would be compilable as resource-only DLL's so this idea would actually increase AutoIt's size instead of decrease it.

Then we have the issues with compiling. Quite simply, there can't be any trace left of AutoIt on a machine, in my opinion. That means if AutoIt has a fatal crash (Hard, not soft), it will have to somehow cleanup all the DLL's that it needed. The solution for that would likely be "embedding" the DLL's into the EXE and running them from there. That would mean we'd have to find the DLL's in the EXE, map them to memory and fix-up the import tables in the EXE to point to these modified addresses. This keeps the files off the hard-drive.

These are only the issues I can come up with off the top of my head and the best solution I currently know about for the presented problems. Who knows what else I've not thought of.

Have I scared you off the idea yet? Now do you see why my saying that multiple AutoItSC.bin would be the only viable way? I've dabbled a little bit in some of the aforementioned concepts and after a day or two of playing, it begins to feel a lot like work.

[MSLx Fanboy]

Honestly, I don't understand why the filesize (as small as it is, comparitively) is such a big issue.

I just finished working on a 160 line project for Message, and it comes in at only 156kb, uses very little memory (courtesy _ReduceMemory()), and very fast and efficient (except for that darned gsprint.exe program).

How much of an impact does TCP functions truly have on file size anyways? I've started using TCP functions at my school (student tech team ), and it works great for error reporting and keeping kids on task (very easy to send a processclose command) by keeping them out of popular shoot-em-up games and what not.

TCP functions allow for better inter-script and inter-computer communication, without the overhead of downloading a webpage, parsing it by hand, and running the commands through there (don't get me started on webpage caching )

[Angel]

Great explanation as always, Valik!

I see your point of having different AutoItSC.bin files. That is why I suggested only having 2 or 3 "features" that can be enabled or disabled, but even 3 would require 9 different AutoItSC.bin files.

This seems like a tought problem to solve, even toughter than I thought!

You actually mention something that might be an acceptable solution, though. The problem seems to be that:

1.- AutoIt keeps getting bigger and people don't like that

2.- When using FileInstall files are created in the hard-drive which might not be acceptable.

So is there any way (as you kind of mention) to be able to include a file into the autoit executable and use it without actually extracting from the exe? Kind of when an EXE uses one of the many icons that it includes. Is it even possible to do this in windows? if it was then this issue could become moot...

Cheers,

Angel

[quaizywabbit]

as I look at it, I ask myself " am I getting the Best Bang for the Buck?"(figuratively speaking)

you could up the size to 10Mb and I'd still say Yes!! Why? cause the darned thing works! and I don't have to be a Rocket Scientist to understand it and implement it.

I'm wondering where the future of AutoIt is going towards, and what can and can't be expected and(gulp!).......when ( i.e. design priorities and goals)

Anybody got a clear view of what's really coming down the pike? Or are the dev's just doing their own thing? I have nothing but praise for all the dev's here: Things have made quantum leaps since I joined in February, and I look forward to every Beta release(Jpm......you Rock, dude!) when I get home from work. Seems like it was rather chaotic how it got from then to now, with things like structs opening up huge previously blocked avenues for dll calls, and Com, with 'embedded activex'

capabilties.......jeez! my head is spinning!

Is there a Plan? Jon?

Edited July 13, 2005 by quaizywabbit

[layer]

Half of what Valik said I understood But from what I did understand, I'd like to see the concept of embedding files into an EXE instead of extracting them, into AutoIt. And even though I don't know what plug-ins really are, from all this talk of them, I'd love to see support for them as well.

EDIT: Oh yea, and why is the filesize such an issue? I mean you're not going to be able to keep AutoIt small in size for ever you know! :">

Edited July 14, 2005 by layer

[Valik]

2.- When using FileInstall files are created in the hard-drive which might not be acceptable.

So is there any way (as you kind of mention) to be able to include a file into the autoit executable and use it without actually extracting from the exe?

Thats definitely not what I was saying at all. There is a fundamental difference in trying to execute a DLL located only in memory and trying to execute an unknown file type in memory.

[jftuga]

What if you had 2 versions of AutoIt -- a lean version and a fat version? The fat version would include TCP, other extra functionality, etc. and the lean version would be similar to the current released version. The lean .exe would be ~150kb and the fat .exe, as a guess, would be 200kb. This way, you would not have to have extra .dlls, you would simply use the fat version. Maybe the AutoIt-to-Exe compiler would be smart enough to decide which one you needed, too.

Would two builds be too much (or too hard) to maintain?

-John

[redndahead]

EDIT: Oh yea, and why is the filesize such an issue? I mean you're not going to be able to keep AutoIt small in size for ever you know! :">

<{POST_SNAPBACK}>

The original intention of AutoIT and still my intention for using it is to script installs or changes on the computer. File Size plays a huge importance because the bigger it is the longer it takes for my scripts to run across a network. Believe it or not AutoIT isn't just for local computer scripting tasks.

red