Integrated function _FileListToArray improvement?

[Tlem]

It is doubtless that you are better than most of the members of this forum, but in your speciality...

It is doubtless that certain members of this forum are sharply better than you in their domain.

The respect for the others is a domain where little of us let us are better.

[Zedna]

Zedna, it's frustrating. People aren't happy they can do something. It must be built-in or at least in a UDF distributed by us. It's absurd how some people can't use code unless we are somehow involved with the distribution of it.

In contrary, it's good to have AutoIt also with its UDFs under (neverending) improvement/development and let it be better and better.

I like to participate in adding new API functions or improving some existing (UDF) code so I don't understand your attack over Tlem here.

I think he is doing good name for Autoit through his France Autoit forum. He is not some bad kid moaning here, he wants to help.

If his attempt is not right, then OK, but it could be said fairly.

In my eyes you are not bad Valik :-) But your responses over the forum compared to other devs/moderatos are not so friendly.

Edited June 16, 2009 by Zedna

[Valik]

The following tickets exist for _FileListToArray(): #973, #966, #834, #820 and #599. All of those are Feature Requests. That's not counting the miscellaneous minor bug reports or changes to other functions that directly impact _FileListToArray(). More than five tickets and numerous forum threads for one function. That is excessive.

I've been programming long enough to know that everybody wants to reinvent the wheel. Hell, I don't even use _FileListToArray(). I have no less than two file searching functions. But here's the problem: We can't be a science project. We need to have a stable, functional version of the function. We can't be rewriting it every 6 months. It needs to be stable. Stability comes at a price; usually the price is performance since it's better to keep a known quantity than rewrite from scratch or make significant alterations. We must provide a stable core so people may use what we provide as the building blocks for their programs. That's the thing nobody understands, though. Everybody wants everything built-in or an included UDF.

As for me? I think we've been over that subject many times now haven't we? I see no need to re-hash the same points again and again. I'm a dick. Get over it.

[Zedna]

Hell, I don't even use _FileListToArray(). I have no less than two file searching functions.

In my case it's different. I use some "file listing" routines in MANY of my scripts and sometimes I need also recursion so I use very old _FileSearch() or new _FileListToArrayEx() from the forum.

I think this _FileListToArray() UDF function is extensively used also by other scripters.

I understand your arguments but please consider also ours.

I'm scared from your previous replies because I feel we are maybe near the stage similar with Opt() or _StringAddThousandsSep() --> it's questionable so why not remove it ... I'm said about that.

[Tlem]

@Zedna

Just for the fun. I take a look on your _FilelistToArray function with full path.

This : _FileListToArray(@SystemDir, "*", 4) give me the result in about 4960ms.

If you made my sugestion, it give me the result in 60ms.

I have noticed that the use of BitAnd decrease the speed of the treatment.

Tomorrow, I will test the 3 versions of the function with 500000 files on a directory on the desktop (so the name of path and files will be more longer).I will give you the results.

If I can, and if my PC support that, I will test on 1000000 files.

Edited June 17, 2009 by Tlem

[Valik]

I understand your arguments but please consider also ours.

What is your argument, Zedna, that we are obligated to spend the rest of our lives fucking around with this one function? The last time I checked there were several thousand functions we have to manage. Constantly revising one function is a waste of time.

[Tlem]

Probably to close the topic (I hope so), this is the result of my tests (each test was made 5 times on the same machine) :

I stop the creation of files at 260000 because Windows don't like so much files in the same directory.

If the directory is empty, the creation of 10 files with FileWriteLine take about 6.5ms, but if I have more than 250000 files in this directory, the time increase to 2130ms...

So let's start with 260000 files in the same directory.

Official _FilelistToArray give it in 5250 ms

Zedna _FilelistToArray give it in 6600 ms

The modified version of _FilelistToArray give it in 4150 ms (and no problems with memory).

So the gain isn't significant, but it steel give the result faster.

Now, this is a test for Zedna.

If I use your function with flag 4, the function return the result in 9370 ms.

My proposition of _FilelistToArray give it in 7230 ms (even with the full path, there is no problems of memory and it seem to work like official _FilelistToArray).

If I use BitAnd to test the flag instead of If $iFlag = ... , the time increase for 150ms for 260000 files.

For me, it's clear, I keep this modified version.

[James]

Just to re-iterate. Leave the subject alone.

You guys are really starting to annoy me with this function. Everybody has an opinion on how this function should behave or should be implemented. Shut up and use what you have. We will never see the end of revisions if you want to tweak it because there will always be somebody better than you who can come along and make it "better".

People just need to get it through their stupid heads that we dot not have to do everything for them. If they can make modifications to something they need to shut the fuck up and use it. Or post it on the forum if they want to share. A significant amount of community feedback can drive feature change but not one random person saying "hey I can do that better".

[Zedna]

Thanks for deep tests Tlem.

My adition (Flag=4) wasn't meant as speed optimized but as added functionality.

I copied style of BitAnd() parsing of flag from another UDF function.

So my advice according to Valik's posts is: Make your modified version as new topic/script in Example forum

so people (including me :-)) may easily find/use it if they need faster/better version than oficial one.

Edited June 17, 2009 by Zedna

[Valik]

So my advice according to Valik's posts is: Make your modified version as new topic/script in Example forum

so people (including me :-)) may easily find/use it if they need faster/better version than oficial one.

And then after a few months when it's proved to be stable come back to us and we'll probably add it as long as you demonstrate that it is stable (link to the thread I guess).

[Tlem]

Done.

[JSThePatriot]

What I don't understand is I suggested that in my post on this thread...oh well glad it's done.

Jarvis