KindEbook Price Query (old version)

[TheSaint]

Due to a huge issue I've been having with editing the first post in this topic, since the forum upgrade, which makes it excruciating to modify, I have relocated a sizable chunk to the second post in this topic. I'm hoping that will improve things, and things seem to have sped up a little, but only after fixing all the remaining problem links in the first post.

I may not get a full improvement though, until I fix all the dead links, now located in the second post. This is purely a guess though.

Thus as of this post, I'm still having long lag issues while editing, and to add insult to injury, the fixed links are misbehaving.

For some unknown reason, when you click a link, it takes you to the right location, but then when the page stops loading, it jumps to the end of that page or another location ... very weird.

EDIT
It seems to be a browser (or browser version) issue for the link problem, and it may very well be causing the editing issue too. Oh  joy of joys, something for me to do tomorrow ... update Firefox.

Edited May 15, 2015 by TheSaint
further enlightenment

[TheSaint]

UPDATED - see first post.

(v3.6)

Another minor update while still working on the major v4.0 update.
Private Comments are now supported and separate from previous Comments, which are now called Shared.
Image Data window can now be viewed whether Image Data exists or not.
Renaming of a Title or Author, now creates a respective backup key, and has restore options.

NOTES - When the Image Data doesn't exist, you still get the warning, but not when using UP and DOWN arrows. Private Comments can be viewed in the same manner as Shared Comments, but can only be modified via the right-click option. Shared Comments are not truly shared in this version, but they become fully so in the upcoming v4.0. In fact, having Private and Shared settings in this version, is just a preparatory aid for the v4.0 Updater, to maintain secrecy where needed.

Apologies for getting stalled with this update partway through uploading.

As previously mentioned, I am holding off on completing v4.0 for a while. I may even eventually rewrite it from scratch, though that seems unlikely. So I recommend that users update to v3.6. An eventual v3.7 is not out of the question.

[TheSaint]

About to investigate a BUG I discovered last night (an important issue, especially, for multi users and those who have relocated bought ebooks).

This probably affects selections after adding or removing an ebook entry too, if my guess on the probable cause is right.

If I am indeed correct, then it was a silly oversight by me.

Essentially, I was having selection issues after a specific number of lines, where I would select a line entry and selection would immediately jump to the line below. If I kept going up the list, this issue eventually stopped at a specific line, and only repeated when I selected below that line number again.

I have noticed an issue on the rare occasion in the past, but put it down to a quirk of ListViews after changing the number of listed entries.

On this occasion (last night), I had mistakenly left the Relocated Bought Ebooks user as the currently selected user when I'd closed the program on previous use (I'd relocated an absolute bargain there). I hadn't realized that on startup, and initiated a sort of favorites and then a Query ALL, at which point I did suddenly realize and thus cancelled after only two ebook prices had been checked (which has made me decide I need a prompt for Queries by this type of user). Anyway, I then selected the correct user and tried again. Towards the end of that query, I seemed to notice that a few changes were not being reported in the overall count.

In other words, a change would be indicated by an asterisk, but the count not go up.
Which lead me to investigate further once the Query ALL finished.
It was then, when selecting an entry near the bottom of the list, that I was having the jump to the next line selection issue.

After discovering this and eventually finding out that this behavior stopped when I went up the list far enough, that it dawned on me, that a variable related to list entry count (and selection) was probably not being updated on User change, and that the value was less than it should be, due to the Relocate Bought Ebooks user list being much shorter (smaller count of entries). This would probably not be an issue going from longer to shorter lists and then back again, but as the program had started with a shorter list, then going to a longer list could cause this, if the necessary variable wasn't updated.

How I might have overlooked this variable being updated after a user change, and perhaps even adding or removing an ebook, I'm not really sure. Perhaps I just thought updating the list count was enough or maybe I was sidetracked or interrupted when I did the modifications for having other users etc.

At this point, I have not yet checked my code, to make sure that this is indeed the issue, but I suspect it is.

Stay tuned for v3.7 and a fix (hopefully) and the prompt I mentioned. I am also considering relocating General Comments to its own file, so that they truly can be Shared Comments. That last is certainly a consideration, as it could be some time before I continue work on v4.0, as I want to devote most of my attention right now, on my Audio DVD Producer program.

P.S. It is not beyond the bounds of possibility, that the other issue, which drove me in the direction of creating a v4.0, might in fact be related to this same bug. If that is the case, and I manage to fix it, then I will have even less reason to continue work on v4.0. At the end of the day though, v4.0 is an improvement on v3.x and earlier, so probably still worth doing eventually ... or maybe as a complete rewrite now I have a greater vision and understanding than when I started ... remember though, that this program has become far more than expected or anticipated ... it really was just meant to be a simple affair. A rewrite may involve an overall name change to simplify update issues.

[TheSaint]

Mmmm ... after some preliminary attempts to reproduce the issue (aside from marking a book as Bought and then relocating), I have not been able to reproduce the issue. This means (phew) that the issue may not be exactly what I thought or not as wide ranging, and perhaps only to do with relocating (though Add and Remove may still be at issue too).

This is great, in that I do not need to give myself a big slap ... but perhaps a little one only.

However, this means it could end up being much more complex, and I really don't wish to devote that much time to it right now, as Audio DVD Producer is calling me, and I want to capitalize on the downmix to stereo testing I did much of yesterday.

P.S. I only discovered this selection issue last night, just before retiring to bed, and so aside from a quick investigation I did not devote much mind power to the issue, and was happy enough to investigate further today based on my first suspicion, and take it from there. I find, it is best to not have too much to cogitate over when attempting to go to sleep with a tired mind.

[TheSaint]

Just reporting that I have been sidetracked by other things, including working on my Audio DVD Producer program, and doing my head in from reworking the predictive elements for projected project time and size ... so many variables now.

That all means I have avoided further testing and investigation with the latest issue here.

Despite that, I have still been using the program and making purchases, and not experienced any further trouble. So upon a little more thought, I reckon the issue only occurs in a specific set of conditions, and probably very unlikely ones at that. The prompt I have already mentioned should address that and make the likelihood of the same conditions even more remote.

[TheSaint]

UPDATED - see first post.

(v3.7)

Bugfix for Timer Label causing a miscount of control numbers when swapping users in certain scenario.
Prompt when making a Query from relocated bought ebooks user.
Private Comments are now queried when copying or moving an ebook to another user, if they exist for that ebook.
All Shared Comments are now stored and read from the shared file 'Comments.ini', though they still also exist in each user file for now, if no longer referenced or updated there. Any conflicts during this updating change, between users, meant comments were combined but separated by a pipe '|' character.

NOTES - I finally got the urge to troubleshoot this program last night for the previously mentioned bug, and add some other improvements.
It didn't take very long to replicate the problem, and then discover the cause, which was an errant Label that needed deleting after use before doing some other things. I'd originally intended for this Timer Label to be deleted after use, like the Please Wait one, but had decided it was handy to leave it displayed, not fully realizing the later impact, when changing user etc, due to how ListViews work, and Control ID's. Anyway the issue is resolved now, and I expect it may have also been the cause of the other issue that drove me into pursuing a v4.0.

v4.0 is on indefinite hold now, with both issues (bugs) having been seemingly fixed, plus making Shared Comments truly shared.

P.S. The Timer Label, which displays the time during a Query and the end time taken, is still displayed until such time as a User is changed, ebook added or removed or relocated etc, list sorted and a few other things that require it being deleted.

Edited June 13, 2015 by TheSaint

[TheSaint]

Well, as luck would have it, it appears the store in question have changed something.

The price is not being returned, and on the third item being queried, the program crashes.

This happens with the new version and the older v3.6, so not something I have done ... though the crash is surprising.

I cannot be bothered chasing up what it might be at the moment, but I imagine it only requires a simple adjustment after investigation .... hopefully.

P.S. A quick play with how many HTML characters are read, did not fix things, and sometimes the program crashes on first item queried, so what item seems relevant and not number of query. I'm guessing a key word or it's location has changed ... and maybe more than one. The crash is probably array related and so I may have left out a test or two

Edited June 13, 2015 by TheSaint

[TheSaint]

I've had time to have a quick look, and determined that the info I need for my program to work, is no longer being gleaned by _InetGetSource. Some other elements have changed as well, but chiefly I get all I would need by manually saving the Web page, which creates a html file almost twice the size of that created by _InetGetSource.

I've also tried InetGet but that returns the same data as _InetGetSource, so I need to find some method of downloading the full web page to overcome this problem. I'm presuming the key elements I need are being stored in dynamic JavaScript, which is being overlooked or stripped out by the Inet functions. There has to be some way of replicating what I can do manually.

[TheSaint]

For now, the program is broken.

An example of the issue, is the following.

If I manually browse to the following item on my wishlist.

http://www.amazon.com/dp/B00TE3HFLQ/ref=wl_it_dp_o_pC_nS_ttl?_encoding=UTF8&colid=2R26DRFTGUUA&coliid=I3IK6RKPAQJ4WM

It shows it as being available to me for $13.09 (and I'm presuming that is US currency). I also have a Buy now with 1-click button.

If I use _InetGetSource or a command-line third party web page downloading program, it returns a different scenario.
The ebook is not currently available for purchase, and is priced at $21.31, with no Buy now with 1-click button.

If I manually save the web page of the first instance, the price of $13.09 is in the html source, but is nowhere to be seen in the second instance.

My program will remain broken until I can find a method to faithfully download an ebook's web page data.
I'm presuming this is something to do with JavaScript and Cookies, and that perhaps I need to provide some kind of identity to get the correct return.

Edited June 14, 2015 by TheSaint

[TheSaint]

One thing I have discovered today, is that Amazon are now dishing up pages as https rather than just http. I found that out after logging out and then back in. Now all my wishlist items show https at the start. This alone would have prevented adding new ebooks to the program, as it tests for http only.

However using http does still work, and using https does not overcome the lack of correct price being returned, even when I use InetGet instead of _InetGetSource, and provide my username and password. There is only a few kb's difference in the Html source that is downloaded/scraped.

Perhaps there is a delay occurring before the current price is displayed, and both InetGet and _InetGetSource are returning too soon?

Currently, I'm hoping a DLL (program) I found online will do what I need ... it promises a lot.

[TheSaint]

For now, I have decided to put further development on hold (but see below), as the program is broken (no longer works) and a solution seems nigh impossible or difficult, and in any case, I have no wish for Amazon to be unhappy with me (not that they should be).

You can read my Web Page Price Checker topic in Chat to see further and related discussions. The latest entry by me is here. Link not working, try this and browse to the entry.

Further to that, my program can still be useful, to gain access to ebook pages faster than going through your Amazon Wishlist, so in that regard you can still be better off ... still a lot of clicking and waiting though.

I will probably do another update, so that you can still add ebooks from your wishlist to the program, but Prices won't work.
I am also considering making it quicker and easier to open an ebook's web page.
That last, would mean less clicking and no right-click menu selection. So perhaps a button and/or checkbox, with the ability to auto move to the next item on the list.

A true shame that this has occurred.

P.S. The damn specific Post link above, is not currently working. On Jon's advice, I've now pasted the link directly into the page and are bypassing using the Link button from now on, until the forum issue is fixed.

Edited June 15, 2015 by TheSaint

[TheSaint]

I'm also looking at the option of the user copying and pasting the current price into the program themselves, and the program processing it ... perhaps keeping a short term record as well, and providing a Report for all recent changes, at the click of a button.

That would mean that the program still essentially works, if in a much more convoluted way.

[TheSaint]

UPDATED - see first post.

(v3.8)

Program is broken as far as getting prices automatically from Amazon, but a 'manual' mode has been added, and set by default, but can be reset to 'automatic' in program settings. The manual mode allows you to do your usual Query (one or ALL), and presents you with an Input box to manually paste the price you have copied from the ebook webpage that has been executed. You can stop at anytime, by clicking the Cancel button. The usual Report will be displayed and any changes stored.
NOTE - You cannot currently ADD any new ebooks to the list with this version, but I will address that issue soon.

NOTES - If the program is not running with the On Top setting enabled, then it will be enabled at the end of a Query ALL, to ensure the window is visible when restored. During a Query ALL, the main program window is minimized and a floating Input Box appears top left of screen for each item queried. Clicking Cancel will exit the Query All process. The floating Input Box relies on an executable called CheckForWin.exe, to ensure it is set on top by the program. Before the Input Box is presented, CheckForWin.exe is executed and the URL for the currently selected ebook is executed, and the Amazon web page should appear shortly afterward in your default browser. You then highlight and copy the current price from the webpage, and paste it over the previous current price displayed in the Input Box. You then click OK, and if you wish to, close your browser or browser tab for that ebook.

If an ebook price hasn't changed, just click OK anyway, as the program does the usual comparisons, and will skip any matches with a previous current price.

[TheSaint]

Okay, I've now done a full manual Query ALL of 42 ebooks on my list, and it wasn't too painful or take too long. It did however strike home to me, that the ebook title needs to be displayed on the Input Box, to reduce the chance of pasting a price into a wrong entry. So expect to see that improvement tomorrow, as I'm ready for bed now.

I've also had a report that prices are not being detected wrongly for at least one other person, so I will look into fixing the program, even if it won't benefit me.

[TheSaint]

Had a bit of time up my sleeve waiting for something to finish on another PC, so I coded the small improvement I mentioned and just replaced the other version of v3.8 with the update.

[TheSaint]

UPDATED - see first post.

(v3.9)

Program can now have ebooks added to it manually, when in 'manual' mode.
The Timer and Wait labels are now dealt with correctly, by being hidden when not required or wanted.
An Input Box is continually presented for user to manually add an Ebook's details (Title, etc).
No other fix has been added yet, to deal with broken status in regard to automatic price checking.
That last, also includes not automatically getting details when adding ebooks. I may do that next.

NOTES - Adding an ebook manually, still requires an Internet connection, though it is only checked and not used. This is because of the ISBN element, which is a requirement. It is reasonable to expect, that you will need to have the Amazon web page for the ebook, available, so you can copy the relevant detail from it, to paste into the Input Box fields. Clicking CANCEL at any point, will abort ADD and remove any detail provided so far. Clicking OK in the case of Pre-Order or Book Description, without changing the displayed default text, is seen the same as clearing the default text to none. Adding a Book Description relies on a clipboard comparison, to overcome the 254 character count limit of an Input field. It is also then able to deal with carriage returns etc.

And here we are, almost up to the fabled v4.0, which will undoubtedly be a very different beast now.
If I need to refer to the original again, it will now be called v5.0.

P.S. I discovered through a discussion I initiated in the MVP forum, that the way I handled the Timer and Wait labels, was not correct programming. Chiefly, I had used $Label_timer = "" after having deleted the control, and the same for $Label_wait. They are now both always created, but hidden and shown at need. I also learnt how to better use Listviews, but incorporating that into this program, may never happen or have to wait until another time ... perhaps when it is working fully again. I have however taken on board the new knowledge for the future. I'm not overly enamored at the added complexity though, despite all the obvious benefits. Thanks go to Melba23, guinness and jhcd etc.

[TheSaint]

UPDATED - see first post.

(v4.0)

No, not the mythical v4.0 .... that is now v5.0.

A Query ALL can now start at the currently selected entry, via a prompt.
An option to auto close current browser tab (using Ctrl + W), has been added for when getting price in manual mode.
Input Box window widened slightly in ADD Ebook manual process.
A delay option has been added, to allow a pause for specified seconds between queries during a Query ALL. This delay may be helpful in preventing erroneous detection as a malicious scraper.
Ebook cover can now be a .jpg file, in the Image Data folder, where filename matches ISBN ID.
Bugfix for whitespace in Price in manual mode (sorry about that).

NOTES - Just when you thought this program almost dead and buried, it drags itself up again from the primordial soup.

Added some improvements that would have been welcome in the original fully working automated version. In fact, one of those I added (Delay) is of no benefit, unless it does get fully working again ... so I live in hope.

Close Browser Tab, works quite well for me, and I only had it fail once out of 45 ebook queries (unknown reason, but probably mistiming or an ebook name change). This option, if you enable it, means less clicks and probably faster processing, so adds a bit of automation back.

Starting at the currently selected item, in the current circumstances, can be really helpful as doing a Query All is a much longer process. It does have obvious benefits for automatic mode though, if we can get it working again.

My apologies for any (if any) that had price errors due to copy & paste whitespace. I overlooked that initially and then kept forgetting to fix it. If however you were careful with copy & paste you should not have been impacted.

Not really much more to do now, except the obvious.
If it is indeed possible to get it reworking for some of you, then I am just waiting on a Html.txt file from one of you (created when you add an ebook), to compare to mine. I cannot go forward without that. Mine doesn't have the data I need, whereas yours might.

One of the obvious missing elements of the manual mode for adding ebooks, is the lack of cover.
It is possible, that if you save the cover as a .jpg and rename it to .dat and then give it the ISBN ID as filename, and put it into the Image Data folder, that it might work. I'm about to test that myself.
EDIT - It did not work, so I modified the program so it does, and as no-one has downloaded this version yet, I just uploaded it as a replacement. Any .jpg can now be used as a book cover, provided it has the extension .jpg and the filename matches the ISBN ID (i.e. BOOIJKLDV.jpg).

There is a possibility, that despite not getting a price or correct price, that I may indeed still be able to get all the other data, including Image data. I just haven't bothered to look into that just yet.

P.S. Just for the record, I do realize that the so called ISBN ID is not really an ISBN. I just forgot what ID it really was at the time, and keep forgetting (ASIC or something similar), but saw reference to it again recently. It is kinda too late to bother changing it now, It works and that's the important thing.

Edited June 17, 2015 by TheSaint