KindEbook Price Query (old version)

[TheSaint]

Seems like I need to fix an issue with regard to image data, where it is created even though a cover iamge may not exist (except as a default blank one), and therefore prevents updating when a proper one does eventually exist.

I've just discovered this for a Pre-Order I have, where the book's cover image has only just now been added to the web page, months later.

I had presumed wrongly, that it was only a matter of getting Missing -> Image Data using that right-click menu option.

I tried that, only to be told the data already existed, which it turns out was just default blank image data.

I need to add in a replace query prompt it seems, as it was a scenario I had overlooked, and not the original purpose of that Missing option (showing any image was not a feature of the program in the beginning).

I'll probably code something soon (tonight possibly, but more likely tomorrow morning).

Edited April 22, 2015 by TheSaint

[TheSaint]

UPDATED - see first post.

(v3.3)

Changed the 'Get Missing -> Image Data' option, so that any existing file can be replaced on OK from a query prompt dialog.

NOTE - A quick simple update for a discovered and necessary missing feature previously overlooked.

[TheSaint]

What a difference a day makes.

It has just taken me less than 4 minutes to Query All my ebook prices today, while yesterday it took over 11 minutes for the 42 items on my list.

Clearly, my Internet connection is crap on certain days and or times.

Which I guess is not really surprising considering I'm about as far away from the exchange as you would want to be with ADSL 2, and still have it usable, though never getting anywhere near a good ADSL 2 speed.

On a good day, I can download a GB in about an hour.

Yesterday, it must have been pretty bad, as I even had a failure to read the current price for an ebook, on two separate query occasions, and it being a different ebook each time. That has not happened to me before, since using the program, all these months, that I can recall.

[TheSaint]

UPDATED - see first post.

(v3.4)

If the current price for an ebook entry gains an asterisk during a Query, then this now persists after all but a program closure. This means that you can sort or change user and later return to the original user and any asterisks will be restored.
Added a hash (#) indicator in the Current price column, when either a web page is not found or the price cannot be determined, during a Query. The hash does not persist after a sort, change of user or program closure, etc.
Time Taken for a Query is now shown on the Report dialog and in the temporary text file.

NOTE - Just minor improvements to make use of the program a little easier, by being more informative.

I have not been able to do much testing as yet of these little changes, but other than the possibility of one line too many on some screens (dependent on settings set by the user) in the Report dialog, which can be adjusted, nothing else should be impacted (or the possibility thereof).

[TheSaint]

A possible bug found ... or as they say, discovered but no reason yet found.

The following happened to me a couple of days ago, and I've not been able to replicate the issue result again yet..

I did my usual Query All with the program, and when it finished I was presented with the usual dialog reporting all changes in price. Only thing is, several items had a huge jump in price, one as much as $20, so to say I was suspicious, is an understatement.

Anyway, they all proved to be wrong, which I double-checked by browsing to the web pages in question.

I then queried each of those items individually, and they all returned the correct results.

So, I'm a bit bewildered, as apart from a Loop and code to move onto the next item on the list, the main code is exactly the same, whether ALL or ONE item is queried.

I checked again today, after restoring the backup preceding the changes, and all went fine and as expected.

Having gone through my code quite deeply, I am at a loss for a reason for this issue, other than somehow having a new price compared to the price of the previous item (which I need to look into more deeply). How that last state of affair could occur, I am not to sure, but I am speculating, as this has never happened before, that it has something to do with an adlib issue that involves timing, as I recently added a timing element. Adlib was already being used, as it was monitoring for STOP button being clicked, so I just added code to that procedure, that updated the timer.

Adlib is calling my function every 250 ms for STOP checking, and initially I just made my timer code act on the fourth call to the function so every full second, but my timer label was not updating in a regular smooth fashion, so I changed it to every third call and then every second, without any improvement. Anyway, I left it at every second call to the function.

It seems that the adlib function call seems to hang, while the main code is querying an item's web page for a price change, and will not update the timer label until each query has finished. This means that instead of just seeing incremental changes to the timer occurring, we regularly get jumps of a few seconds or more after a lag.

That makes me suspicious when it comes to my issue.

I guess time will tell, and the reports of the same problem by others.

It has left me thinking that I need to add a Restore Backup option, to simplify that process, and possibly include the 'Settings.ini' file in the backup process as well. So expect to see that in the next update.

Edited April 27, 2015 by TheSaint

[TheSaint]

Mmmm ... I'd nearly finished the Backup Restore code, when I suddenly realized that things were more complex.

I've made the Timer an option now, and not set by default.
I will probably add a warning when setting it, for the illusive issue/bug, as once again today, I had no problem using it.

The complexity with Restore, involves the fact, that backups are only created on program start and during the Backup enabling process. I have catered somewhat for that, with ADVICE and a WARNING, but none of that takes into account, that any ebooks added to current active user file or removed, plus any changes (prices, favorites, relocations, etc), may be lost by restoring a backup (this means good changes as well as any bad). This is something I need to look deeper at and cater for, especially in regard to ebook existence on a list being lost. It may even mean creating a separate Catalog file just for ebooks and regardless of the User files, but which they all refer to when starting up. Perhaps username will become a value for each ebook in a key called usernames (pipe delimited)? I imagine this will mean duplicating a lot.

IMPORTANT
Until I sort this out, I recommend Users of the program consider the above issues when they are contemplating using a Backup ... as you should anyway. Backups are a bit of a safety net not to lose all, but with the current issue unresolved, might become necessary. This means, that some manual editing may be required ... a tricky tedious operation by all accounts. It would possibly just be better to individually re-Query any problem entries. If the program hasn't been restarted since, then no bad backups should have been created, only just a Report record (if enabled).

Edited April 28, 2015 by TheSaint

[TheSaint]

ADVICE for those currently using this program.

I am in the midst of writing an updater, that will be a separate download, along with the new version download of the program.

This means, that the next version of the program will be jumping to v4.0.
This is to signify a clear distinction between old and new, as I will be rewriting a significant portion of the program code, to cater for improvements, that will assist with restoring a backup, so that important entries are not lost during restore.

If any user is still using a very old version of the program, that uses the 'Catalog.ini' file, then they will need to update to a newer (pre v4.0) version of the program, before using the Updater (also see NOTES below). This is because, User names are necessary for the updating, and a newer version of the 'Catalog.ini' file will once again be a part of the program.

Essentially, all ebooks, regardless of User, will now be stored in the 'Catalog.ini' file, as well as in the User INI files.

During a Restore of a Backup, these ebook entries will be cross-checked, so that nothing is lost for a User.
However, if an ebook has been Added or Removed prior to a restore, then those changes cannot be undone, only price changes (if in a backup). This is because, only ebook names, authors and starting details (prices when added) will be stored in the 'Catalog.ini' file. In fact, the only time the 'Catalog.ini' file is written to, is during an ADD or REMOVAL or when another User starts sharing the same ebook. At all other times, that file is only used for cross-referencing.

ADD or REMOVAL is permanent, and the 'Catalog.ini' file takes precedence over User files, except where Users share an ebook, in which case the removal or addition is only permanent for the User that instigated it.

This all means, that the 'Catalog.ini' file has an entry for each ebook of the user(s) that have it listed.

NOTES - The current version of the program, has some previous updating elements, which will be removed from v4.0 of the program, as they will no longer be necessary, with this fresh start. I will be leaving v3.4 available for a time, for those who may need it, before using the Updater.

Edited April 30, 2015 by TheSaint
continuing

[TheSaint]

Updater has been finished and run, and seemingly worked as desired.

Time for me to relax now, and watch something.

Tomorrow, Dog willing, I can start re-working (adapting) the original program code to utilize the new approach setup by the Updater, plus continuance of the code I had nearly finished, for what was going to be update v3.5, now v4.0.

[TheSaint]

Made some minor additions to the Updater today  - Settings.ini and new Catalog.ini files get a copy placed in the Backups folder.

I've done a fair amount of the necessary work and adaption to the program itself, and so v4.0 is well on track, though doubtful i will complete it today (other fish to fry). Currently you can start the program and switch users and all works as it should. I've finished updating the Backup procedure, which sees the Settings.ini and Catalog.ini files getting up to five backups like the User INI files, based on file modification time and cycled so that the oldest gets removed and the rest shuffled along and the latest backup file being added.

Currently any missing entries in a user's INI file are restored on program start or when switching user. This only occurs for those entries that are listed as belonging to the user, but are now missing (presumably due to restoring a backup file).

The biggest thing I still need to do (complexity wise), is to get the Restore a backup file procedure, doing exactly what it needs to, so that nothing that shouldn't be, is lost. All except for perhaps the latest price changes ... dependent on whether a backup includes them or not. I'm approaching this as a restore requirement, because errors occurred during a Query, as being the most likely reason for restoring a backup file. With that in mind, you wouldn't want current prices to be restored or what would be the point. You'd want the previous prices restored, presuming they were correct at the time they were backed up. The only issue with this, is that nothing is backed up while the program is running, as it requires a restart of the program, so if multiple queries have been done since the program started, and the last erred, then a backup restoration would by necessity replace the results of all those queries.

That last, should not be too damaging, as an immediate query of ALL after the restoration, should get things back to where they should be pretty much, with only the time and date being a bit out compared to the originals. Elements like assigning Favorites, Bought, Pre-order and additions and removals of ebooks, should survive a restore. No Image Data files or summaries should be impacted. If anything, a restore has the possibility of leaving orphans entries in a User INI file, which I might make a cleanup process for.

Edited May 1, 2015 by TheSaint
continuing

[TheSaint]

I came across some possibilities today while working on the next update, that meant I needed to rewrite a sizable chunk of the Updater ... not so much for me, but due to plausible scenarios of other users. I'm still at work on this, and obviously the update is on hold until I have finished.

Not helping matters today, was an old PC (blue screen) issue that happens at random (sometimes a few months apart), and feeling like all mental processes are happening through a fog or mud blanket ... which of course, meant I didn't save before the crash (forgot my five minute drag a copy regime) ... luckily the script wasn't corrupted ... so no wasted coding, just wasted time and a headache. The mental fog didn't become apparent until this happened of course, and I've felt like going back to bed ever since, but have labored on despite feeling like crap. Carrying on helps make the crap feeling less apparent, except when I struggle with some complex logic, that nearly does my head in. Being wise to that last, I just work on less complex aspects whenever that occurs, and leave the other until later (tomorrow no doubt), and so I am proceeding more slowly than I normally would.

[TheSaint]

Things often don't go exactly to plan, and so it is with the latest Update and Updater.

I'm still fiddling with the Updater ... only because I had some more thoughts to make it better, which occurred to me after I finished the latest version yesterday, and I've had to deal with other issues today, so nothing more may happen until tomorrow.

Somewhat ironic, that the issue that drove me to this major update, has never happened again since the first and only time.
So I may be expending all this effort without any real need to, though there will be some good improvements and benefits when done.

So, was it some weird glitch that happened to me solely, and something to do with my Netbook's memory perhaps?

I've posed my theory of possibility previously, and I remain none the wiser now, as despite still using v3.4 (latest version) of the program twice or thrice a day since, it appears to be working flawlessly ... and it must be more than a week later now.

One day, I will get back to work on some of my other programs, especially the DVD Audio one.

P.S. As always, I'm interested to hear if anyone else has experienced the same Price Errors issue or other, with v3.4 etc.

[TheSaint]

Finally finished the Updater (I think), and started work again on the Update.

All subject to change of course, as I wrestle with should and shall and do I.

[TheSaint]

Just took 3 min 11 secs to Query 47 ebook prices.

Must be a good connection speed for me right now (for a change), as that is the fastest Query so far, that I've noted.

Edited May 5, 2015 by TheSaint
correction

[TheSaint]

Wow, I just did another Query before logging off for the night, and it took only 2 mins 53 seconds for the 47 items.

I hope this is an improvement to my connection, beyond today.

[TheSaint]

Another goodish connection speed day.

Well, the good news is that I have not needed to redo the Updater again.

The bad, is that this process is taking far longer than expected ... not helped by a headache yesterday and some mind fogs in recent times.

Anyway, as much as things pretty much work the same in the program, I am now having to cater for and use the (master) Catalog.ini file. This means I must ensure it is modified whenever it needs to be along with the usual user file keys and values.

One benefit so far, is that if another user has indicated a shared item is bought, you will get a splash displaying briefly to tell you so when you select it. This even caters for multiple buyers of the same item.

I've been working on the new update in fits and starts, but much has been done and I am making progress.

Aside from that, v3.4 continues to still work for me without issue, so I guess there is no hurry ultimately for v4.0.

Edited May 6, 2015 by TheSaint
greater clarity

[TheSaint]

A new day and a new thought.

It occurs to me, I could just have implemented a simple restore system for a v3.5, that could have been done relatively quickly and simply, based on the idea that a once off separate backup is made every time Query ALL is started.

This would mean, that if you do a Query, and you detect errors afterward, you could just roll back the prices only, rather than restore a complete backup file. There would be some tricky elements to consider, such as individual entry queries, and how long that once off backup should be usable, but easily doable.

Why didn't I think of this simpler solution before?
Well, I'm pretty sure I did, but I realized an overall better solution was required, and so I just went for that, which in the end has turned out more complex than my original estimation, so now while doing the v4.0 update and having to consider various things, that should be done, I am tempted to do an intermediary update. Particularly, as this once off backup method, will indeed now be a part of v4.0 anyway, so not work or code for nothing ... it needs doing regardless.

So I am tempted now to do v3.5, and if I have a backup project file after v3.4 that doesn't have any major v4.0 changes, only the ones originally slotted for v3.5, then I may just do it, so stay tuned. How close I am to finishing v4.0 will factor though, so I will need to estimate that first.

Sometimes you just need to laugh at life and enjoy its vagaries!

Edited May 7, 2015 by TheSaint

[TheSaint]

Not a good connection speed today, as it just took me 6 mins 11 secs as opposed to the fastest yesterday of 2 mins 51 secs, for the exact same 47 items. Afternoons and especially evenings on a Friday, are generally a poor connection speed.

Still working on v4.0, but I did backport the necessary code to a v3.5, which was easy enough to do, and I'm still testing, so expect to see it made available soon, probably via my Google Drive and Bitly, as the download counter here is still not showing anything, and I'm intending to eventually relocate all my programs anyway.

EDIT

It seems I have not been treating my back well enough, so it is giving me hell today, after a little trouble last night. This means I need to avoid prolonged sitting. So I will take a rest from programming for a while until well enough.

Edited May 8, 2015 by TheSaint
elaboration

[TheSaint]

UPDATED - see first post.

(v3.5)

A minor update while still working on the major v4.0 update.
The timer being used and displayed during a Query, is now optional.
The 'Settings window has been shortened in height and doubled in width to accommodate new and future features.
A 'Restore after last Query' right-click option has been added, to return prices and related to their previous values.
Selected Entry Detail (text only) and Selected Entry Detail (image + text) have both been relocated to a submenu called Selected Entry Detail, and renamed to be suitable.

NOTES - Due to some weird bug I've only experienced once after much use of v3.4, I have been working on a major update (v4.0). As a stop gap until that is finished, and because the issue or bug which gave price errors for me after a Query, has never happened again thus far, I have created v3.5. It seems to me, that this elusive bug is probably related to the new timer element introduced in v3.4, so that is now optional, and I have provided a basic limited restore option, that can be used immediately after a Query, to return things to the way they were, if price errors are noticed. The restore option may be necessary, because using one of the optional backup files, does not cater for added or removed ebooks that may have occurred since the last backup. The new v4.0 is set to correct that when completed. Thus far no-one else has reported the elusive issue, so continuing to use v3.4 should be fine, but upgrading to v3.5 is recommended for better peace of mind.

Edited May 8, 2015 by TheSaint

[TheSaint]

v4.0 is almost done, but need to keep resting my back.
Despite that, I put a goodly amount of work in yesterday. Mostly just checking left to do.

In the end, I did some minor changes to the Updater a few more times.

Some of the holdups were just decision making really, and they have resulted in some additional improvements.

One of those, is a split of Comments into Shared Comments and Private Comments. I spent a fair bit of time muddling through that and how best to deal with the past and then the future. It seems simple, but I kept changing my mind and going for different approaches.

The short of it, is that past comments per user, were both shared and private because each user had an individual record, which is still maintained, but having a master Catalog.ini file changes the approach significantly. The idea, is that important comments can be shared, whilst giving the ability for having a private element, and having all backed up properly.

Shared comments in the past, meant shared when copied to another user. Once copied (or moved), there was no longer any sharing. That is no longer the case, as there is now a shared key.

While I say Private, both as in the new key and in the past usage, nothing in the program is really private, as anyone can select another user and see all for that user. In that regard I am contemplating a password arrangement, though checking another user's comments seems like a trouble not worth engaging in, so probably highly unlikely.

However, one needs to consider the likelihood of needing such secrets.
I'm guessing that few (if any) have any secrets related to their ebooks comments.
One scenario I guess, might be a birthday reminder for a surprise purchase, as you can gift Kindle ebooks to another Kindle user. I'm guessing there could be others.

Having Shared and Private comments caters for the future pretty easily, as each are in the Catalog.ini file, with Private being duplicated in the User's own INI file.

Dealing with the past is not so simple, when creating the Catalog.ini file from data extraction of all user INI files. For starters, one doesn't have any indicator of what might be private and what not. Then there is the possibility of different comments per same shared ebook.

I've finally elected to have any comments for the first user listed per ebook, as shared and then any conflicts serving up a Query GUI, for combining or replacing etc. I'm working under the presumption, that the user most likely to be using the Updater, is the first user listed, and no doubt the first person to obtain the KindEbook program. That person, can of course, use v4.0 to correct the state of any of their comments that need to be private, immediately after having run the Updater. Alas, that user may be privy to the comments of all other users, who can also later correct things with v4.0. That last cannot be avoided, at risk of important comments not being shared. Admittedly, these comments are not many in all likelihood. One shared comment, is the Publishing Date for a Pre-order.

Edited May 12, 2015 by TheSaint
corrections

[TheSaint]

Unsurprisingly I guess, I elected to create a v3.6.

Chiefly, because it will ease the transition to v4.0 for some ... possibly, as it features Private and Shared comments, plus some other new things or improvements.

No idea how many have downloaded v3.4, because of the Forum update, and have had no downloads of v3.5 so far, according to bitly ... except the one test I did, so I know it works.

With the release of v3.5 and now (shortly) v3.6, I feel no hurry to finish and release v4.0.
Especially as I still have had no repeat of the issue which gave rise to the idea of a v4.0.

Details of v3.6 and the download links will be up soon.

EDIT

My comments here in Chat are probably worth a read by my program users or budding ones.

Edited May 15, 2015 by TheSaint
greater clarity and further enlightenment