KindEbook Suite (BUGFIXes)

[TheSaint]

KindEbook Wishlist updated to v7.6, see Post #2.

(v7.6)
Some additional processes now have the Splash made optional, based on the similar Query setting.
Exit on a Removal or Relocate is now also possible, via a prompt, rather than waiting for the list to be repopulated (sorted etc).

PLEASE BE AWARE that the following complimentary programs have also been updated here today ... see last few posts for details.

AZWPlug
Add Book & All Formats To Calibre
Batch Ebook Convert
UpdateFromOPF

[TheSaint]

Here are two more complimentary programs.

Authors To TAGs v1-9

Authors To TAGs v1-9.zip

Program Information

Spoiler

This program reads the content of an OPF file used in Calibre.
It may then modify that content and tell Calibre to update it.
Essentially the program seeks to put all Author names into the Tags (subject fields), for the related ebook.
If multiple authors are found, then the Author (creator fields) may also be adjusted. With five or more author names listed the process is automatic, but less than that it is queried.
The adjustment, is generally a single author called 'Various' being created for the sole Author (creator field), and all the other author names relocated to Tags (subject fields) and also to the start of the Comments (comment field).
It is recommended that any existing Tags are not kept, so that you can gain the utmost from the new Author Tags.
NOTE - Tags are often abused by authors or publishers, so their usefulness is problematic or debatable anyway."
© January 2020 - 'Authors To TAGs v1.0' created by TheSaint.

(v1.1) Enabled command-line usage.

(v1.2) Additional checking for Calibre running. Further improvements for re-doing/fixing 'Various', including some HTML replacements for the Inputbox box other authors query.

(v1.3) Further improvements for author name detection, extraction and fixing, including L,F to FL.

(v1.4) Implemented my own type of InputBox (a GUI really) with many more helpful features and no text limitations, to be used when dealing with 'Other Authors'. This is a kind of Bugfix for the InputBox character count limit.

(v1.5) Added more helpful buttons for 'Other Authors' and improved other aspects for them.

(v1.6) Changed the options for TAGS, so that they can now also be relocated to the Comments (Description) field. Bugfixes, improvements.

(v1.7) The currently set default calibre library path is now detected for browsing. Improved handling and bugfixes for HTML elements. The 'Open Folder' button now has right-click options.

(v1.8) Added an ampersand '&' button, which also has CTRL and SHIFT options. CTRL = Undo last change. SHIFT = Search for ' by ' in text from cursor position and select. Vastly improved HTML removal.

(v1.9) Ebooks that are skipped are now also recorded in a Skipped.txt file, and the advise dialog now auto closes after 9 seconds if processing via command-line (for batch continuation support).
 

 

Fix Author Names v1.5

NOTE - Some elements of this program have quite handy usage outside the KindEbook Wishlist Suite and ebooks.

Fix Author Names v1.5.zip

Program Information

Spoiler

Program usage is primarily as a companion to my other Calibre assisting programs and Calibre itself. It only does a couple of basic things, similar to parts of my programs.

i.e. Authors To TAGs, UpdateFromOPF

If those programs are used correctly, then Fix Author Names should not be needed, except maybe for ebook entries that preceded their use or need correcting from earlier versions.

Fix Author Names essentially does three things.

1) Removes a trailing period (fullstop) from the end of the list, if it exists.

2) Turns Lastname Firstname to Firstname Lastname, for one or more entries separated by space ampersand space.

i.e. 'Smith, John & Peters, Mike' are changed to 'John Smith & Mike Peters'

3) Every author name is enclosed in curly brackets, ready to be added as tags to calibre.

i.e. 'John Smith & Mike Peters' is changed to '{John Smith}, {Mike Peters}'

NOTE - The names are not required to be encased in curly brackets for calibre, but I use them to diffentiate them from other types of tags. In fact, I generally relocate other tags to the Comments (Description) section of a calibre ebook entry (metadata).

That said, I do allow some other types of tags, which I also encase in the curly brackets, which might be something like {Silo Saga} or some other kind of collection name. This is particularly the case, where you have sub series or similar. An example of that would be Terry Brooks' Shannara series, that consists of many sub series. Effectively you can only enter one series in the calibre field for series, and so adding other series as tags is a way to overcome that limit. In that scenario, I would list the sub series in the series field and the main series (i.e. Shannara or Shannara Chronicles) as a tag for each ebook. When you then select that tag in calibre, all the relevant ebooks and sub series will then be displayed.

(v1.2 UPDATE)
'John Smith, Mike Peters' is also changed to '{John Smith}, {Mike Peters}'
'UNDO LAST change' button & process added.

(v1.3)
Titlecase button and process added.

(v1.4)
Uppercase button and process added. Most buttons moved around to suit a taller window and new features. An 'On Top' button and feature were added. Info button added. Text copied from below the cover image of an ebook in your Kobo library, can now be converted to a suitable 'Save As' filename when downloading that ebook (if DRM Free). Removal of illegal filename characters supported, plus report on length of filename if over 255 characters. The position of the program window is now saved on exit and restored on restart.

(v1.5)
Leading, trailing or central 'The' can be relocated one way or the other.
i.e. 'The Big Deal' changed to 'Big Deal, The' or vice-a-versa.
or 'The Big Deal - Opening Chapter' changed to 'Big Deal, The - Opening Chapter' or vice-a-versa.
 

 

Edited April 28, 2020 by TheSaint

[TheSaint]

AZWPlug updated, to v6.5, see Post #3. This includes all of AZWDrop v2.9 (source & exe).

(v6.5)
Added a calibre library check to open correct folder after adding to calibre.
Added a right-click List option 'Selected Entry -> Get Detail', to obtain information about an ebook from my companion program 'KindEbook Wishlist' or online via Calibre.
Count on the Taskbar icon should now be correct when adding to Calibre.

[TheSaint]

KindEbook Wishlist updated to v7.7, see Post #2.

(v7.7)
The MOBI-ASIN can now be used in the Search field or copied when an entry selected.
The 'Summary' and other details field on the Image GUI window has now been split in two, and the 'Summary' has an update button for saving if modified.
Improved sizing for some images (JPG only, not DAT).
CTRL can also be used to stop the minimize with the clipboard button (if set to).

[TheSaint]

Batch Ebook Convert updated, to v3.5, see Post #41.

(v3.5)
Bugfix for a zipping error, involving modifying the UDF in an attempt to catch the error and not remain trapped in a perpetual program loop.
NOTE - The original loops were poorly coded in the UDF, with no escape check, so I have myself added some in, but I am no expert, so some of what I have done with my limited understanding, is aimed more at prevention than cure, which means there is room for failure still.

PLEASE FEEL FREE to improve upon my modifications to the ZIP UDF, and provide such or notification of such, here.

[TheSaint]

UpdateFromOPF updated, to v6.3 , see Post #5. Finally added three new screenshots (Dropbox, Editor and Fetch).

(v6.3)
The program now deals with and supports adding decimal numbers in the Series.
NOTE - This involves using CTRL or SHIFT with the INC button, if needing to increase or decrease.

Edited May 11, 2020 by TheSaint

[TheSaint]

UpdateFromOPF updated, to v6.4 , see Post #5.

(v6.4)
Improved issue checking & reporting for Author name, now indicated by red text.

NOTES - Finally got around to checking on and reworking the logic in some of my Author name code, that I did not get quite right during the v6.2 major update. The code is simpler and far more logical, and now incorporated fully as a function, and used more efficiently in other areas too.

[TheSaint]

Tuesday 26th May I started work on a new companion program for this Suite - Get Blurb For Calibre

I've had something like it in mind for a while, but finally started to think more seriously about it a few days ago, and as always, it is a program to fulfill a need.

It is currently still a work in progress. NOW FINISHED, but I still need to make a small supporting adaption to my UpdateFromOPF program.
UPDATE - It proved to be not quite as small as anticipated (often the way), to adapt my UpdateFromOPF program, but it is all done now, bar the testing.
I also added some more code/features to Get Blurb For Calibre, to better support my circumstance and support for UpdateFromOPF ... essentially, I am working across two PCs, whereas most users probably won't be - one has my full Calibre library (offline), the other does the online queries. So the VIEW RESULT button has a CTRL key alternative, which is to copy everything from the program folder to another location (i.e. portable drive), for transfer to my online PC and vice-a-versa. I'd also overlooked the need to extract the updated blurb after UpdateFromOPF use, to show in the Blurb field of Get Blurb For Calibre, on its return ... and for later checking.

Screenshots as the program Evolved

Spoiler

 

Here is a screenshot from Tuesday 26th May.

And another from the following day, where most of it is now working.

And an updated version on Friday 28th May, where an issue was solved, some more things added, and it is almost done.
Just gotta do the code for the two UPDATE buttons, plus create the Program Information dialog.

 

What does it do? What is my need?

Basically, it checks my existing calibre libraries, looking for ebooks that are missing their Blurb or optionally don't have much of a one (based on word count).

The blurb (or description, summary, synopsis, comments), makes it somewhat easier to decide on what to read next etc, especially when you have such a large library of ebooks as I do have.

Unfortunately, when I first started buying ebooks, and eventually started adding them to the cataloging database program calibre, I did not give much consideration to the blurb. To be honest it was all a bit too hard, especially when free ebooks from Amazon etc via BookBub, started coming thick and fast. Later I started to develop my ebook and calibre related programs, and quite some time after developing my UpdateFromOPF program, I discovered how to query online sources for the blurb (calibre command-line tools), and added that feature to that program. I've been using that feature for a good long while now, and any ebooks I have purchased in the last 6 months or more, all have their blurb. Not so my earlier purchases for a few years, where in the majority of cases a blurb is either missing or very incomplete.

So because I now have something like 4,000 ebooks, it is an onerous task to wade through them all, to find and deal with the ones that need a blurb. So I need to Automate It as much as I can, and do it in stages, nibbling away at the huge number of them.

Further Information

Spoiler

 

Get Blurb For Calibre is being designed to use in conjunction with some of my other Suite programs. Like UpdateFromOPF mostly, but also my unfinished program Booklist To HTML, which has not been mentioned in this topic before now, that I recall, and has only surfaced briefly in a Help Forum topic here, some time ago, when I was chasing down an issue with it.  It is a program that creates a series of easy to browse interlinked web pages, that help with deciding which ebook to read next. It gets the text and images etc from my calibre libraries.

Here's a screenshot of the program.

And here's one of the sort of web pages it creates ... though I am not settled on the end result yet for that or indeed the program state itself.

So there is a method and benefits to my seeming ebook program madness.

Some of you familiar with calibre, might be wondering why I don't just get the blurb metadata through calibre? Well, I prefer more control than calibre gives you, for starters, and it would also involve a lot of browsing. My program will reduce that a lot, and UpdateFromOPF in my view is much better and easier to use for getting, editing and updating the blurb metadata, though strictly speaking calibre command-line programs are being used, to do much of the basic work.

Another extremely important factor, for me at least, is where I keep my calibre libraries .... they aren't on a web connected PC. In fact I use an old Win XP laptop. So my new program takes all that into account.

To give a fuller picture, I use my web connected PC temporarily for my ebooks, just to process them after purchase ... enable ownership, set cover artwork, make corrections to metadata, convert to other formats etc, then I relocate them to backup drives and my old Win XP laptop calibre install.

 

 

Edited June 7, 2020 by TheSaint

[TheSaint]

I've now, bar the testing, completed my new complimentary program - Get Blurb For Calibre. See previous post.

Hopefully I will be able to upload that soon and the modified UpdateFromOPF, that now supports it.

P.S. This also means I have been able to resume work on my other new, but unrelated program .... which I chat about here, with some screenshots ... latest post here.

[TheSaint]

My apologies for being a bit slack ... not that anybody seems to have cared or noticed. 

I do have a few updates that I have not gotten around to uploading ... hopefully soon.

Anyway, them aside, I am now considering doing some coding for my Kobo store wishlist. I may adapt KindEbook Wishlist for the third time (JB Wishlist and IonGoG Wishlist), and make a Kobo concentric version ... or I may do an adaption similar to my GetGOG Wishlist program.

At the very least, I may implement a new feature in KindEbook Wishlist for Kobo ebooks that match the Kindle ones listed. This would take a lot less work, and while nothing for Kobo would be displayed on the main program window, you would be able to see the price at Kobo listed in the selected item Details window, where you can also set automated checking after you input the web page address for the ebook or do a manual check. One of the options maybe would be to display a popup if the Kobo price is cheaper.

I was hoping to get BookBub to do notifications of a drop in price for ebooks I add to their wishlist, but alas we have an issue with the way they do things. First off and most importantly, they have just told me that the Wishlist feature only works with the USA stores, and secondly I have things split over more than one country in reality and they don't support that at BookBub. In short, I still use my USA Amazon account for Kindle ebooks ... never relocated my account to the Aussie Amazon store when it eventually opened ( I have some reasons for that), and my Kobo account is for the Aussie Kobo store. As it is, with my daily email from BookBub, if I need to go to the Kobo store, I have to manually edit the link after clicking it for each ebook I want to check out at the AUS store instead ... luckily that doesn't happen often, as Amazon usually provides.

As those in the know have experienced, it is a painful process to browse any sizable wishlist, so my approach has always been to automate that ... one click to check all and then report changes.

So stay tuned.

Edited January 8, 2021 by TheSaint

[TheSaint]

Okay, time for those updates.

KindEbook Wishlist v8.0 - See Post #2 on Page 1 for download.

Spoiler

(v8.0)
Implemented some Kobo fields on the Details window, along with manual and automatic (with QUERY) price checking, selection notification, etc.
Kobo price is shown in Comments, and if less than previous during a QUERY, list text is blue (more is dark red, failure is purple or dark green, black is unchanged).
Label click price toggling.
Bugfix for price comparisons.
Bugfix for Title extraction crash etc.
Warning messages now advise of closure in 4 seconds, with a CANCEL option to keep open. Previously they closed after 20 seconds.

(v7.9)
BUGFIX for a specific type of blank entry, which includes changes to the FIXFILE process.
A right-click 'Check to Remove' option has been added to the 'Selected Entry' menu item.
NOTE - Still being plagued with an elusive corrupt first INI entry, which this bugfix can at least detect now and fix with some manual interaction.

(v7.8)
Added 'Price Query' support for when 'eTextbook' exists instead of 'Kindle'.

 

UpdateFromOPF v7.5 - See Post #5 on Page 1 for download.

Spoiler

(v7.5)
Added improvements to word removal and Rank checking in the Editor window.

(v7.4)
Implemented use of my KindEbook Wishlist program as another source option for Fetch.

(v7.3)
Implemented more HTML text corrections.

(v7.2)
Added a connection check for fetching the metadata (text or cover).
Improved the batch GET option to avoid recording blank description returns.

(v7.1)
Improved the GET metadata for all option of the Dropbox, adding a skip existing to query.

(v7.0)
Adjusted the program to automatically deal with dual authors on Save (no query).
Bugfix for double quotes etc in ebook title file name.

(v6.9)
Implemented more HTML text corrections, and a related query.

(v6.8)
Implemented more HTML text corrections, including with the FIX button.

(v6.7)
Added a GET button to the Dropbox, for batch fetching of metadata for all current library ebooks.
An option to use that saved data has been given to the Metadata Editor window FETCH query.

(v6.6)
Added + and & as dividers for leading 'The' text in title.

(v6.5)
Added command-line support for usage with my 'Get Blurb For Calibre' program.

 

AZWPlug v6.8 - See Post #3 on Page 1 for download.

Spoiler

(v6.8)
Added a RENAME IMAGE FILE right-click list menu option and slide-out panel (for helping with missing Kindle covers on your device).

(v6.7)
Due to an elusive bug, that causes fields for an entry, to gain the previous entry detail (file name excepted), two more right-click options for the database list have been added in the 'Selected Entry' menu entry. These are 'View Stored' and 'Recover'.

(v6.6)
Implemented a fix for the possibility of a pipe character existing in a database ebook entry, along with a replacement of curly for square brackets.
This fixes related INI and List issues.

 

Add Book & All Formats To Calibre v5.0 - See Post #4 on Page 1 for download.

Spoiler

(v5.0)
The Library selection field now has a 'Rename an Ebook' option.

(v4.9)
Minor bugfix for unacceptable character.

(v4.8)
Added a 'RELOAD' button to FIND dialog, to adjust for any changes to the Logfile.

 

Bucket-for-URLs v2.2 - See Post #17 on Page 1 for download.

Spoiler

(v2.2)
Added a PURGE button & process (uses ebooks listing files of AZWPlug & KindEbook Wishlist).

 

---

Lots of improvements. My apologies for the skipped updates ... no doubt due to a variety of reasons/distractions/etc.

KindEbook Wishlist update was a priority for the last few days due to crashing on adding a new ebook to the list. The crash was my fault, but also due to a recent change by Amazon. I have now added the (overlooked) array count check to avoid the crash, as well as code to cater for the new change by Amazon.

Edited January 21, 2021 by TheSaint

[TheSaint]

See previous post for other new updates today.

Batch Ebook Convert has been updated to v3.8. See Post #3 on Page 3 for download.

Quote

Be advised, I am still having an issue detecting when zipping fails, which is quite the random event and I can go weeks if not months without it occurring. It is sure to be object related, but all the tests I have devised just fail to deal with it (detect it) when the issue occurs.

When the issue occurs, the program is trapped in a loop, requiring use of the Task Manager to kill or selecting Exit via program icon in the Tray. When you restart the program, it should detect your conversion(s) did not complete, and so prompt you. You can at this point, just restart Converting again, without having to manually do anything ... and cleanup should occur as normal. I've yet to have the issue occur twice in a row, certainly not for the same ebook.

If anyone wants to look into this, be my guest. What essentially occurs, is that a zip file gets created but fails to be recognized for adding content to it. Without making a single change, restarting the program and re-selecting Convert, will continue with leftover HTML folder and empty zip file without complaint.

I have considered doing some kind of timed query to user, but that just seems b-grade to me ... though perhaps better than being trapped in a loop etc.

(v3.8)
Implemented a manual select (once off) KFXInput CLI option for the selected ebook, found on the Program Options window, and useful where an error reports the need to use such for converting.

(v3.7)
Another Bugfix attempt for the rare zipping error, involving modifying the Zip UDF.

(v3.6) Another Bugfix attempt for the rare zipping error, also involving modifying the Zip UDF in an effort to catch the error and not remain trapped in a perpetual program loop. If the fix fails to work (program stays trapped), then on restart the program will hopefully prompt to process the missing Log file records, plus also deal with left over files etc when conversion resumed.
NOTE - The elusive error may be timing related, so to that end there is now also a 2 second delay between the ebook being extracted to a HTML folder, and the zipping of that folder. This error may also be a permission issue as reported by the error message, perhaps AV related (in which case, the delay may need increasing).

EDIT
This is the error that pops up. And this is despite my code detecting it does exist, and so I am guessing it is some kind of permission issue ... or a bug ... not forgetting I can at times convert many many ebooks in the same way without getting this error ... and size or file count has no bearing ... so an elusive one indeed.

Edited January 22, 2021 by TheSaint

[TheSaint]

KindEbook Wishlist has been updated to v8.1. See Post #2 on Page 1 for download.

MAJOR BUGFIX.

Spoiler

I finally got around to chasing down some oddities I had seen now and then, and to my horror it revealed a major issue with displaying (and perhaps worse) Prices and Summary.

Some time back I had changed things such that Prices and Summary for each ebook were written to their own files. It has been revealed to me that I did so imperfectly, forgetting one rare but important element when it came to reading them back ... what if the file did not exist? Normally the files would exist, and I forgot or overlooked an exception to return an empty variable, which meant the previous ebook Summary or Price would be shown ... oops. Like I said though, it is a somewhat rare case, and only came to light because due to Amazon changes I had some partially written entries due to the crash mentioned in v8.0. To see the issue with the Summary, you would have to have the Details window open and be changing between entries and be paying attention, and of course you would also importantly have to have an ebook entry that had no summary ... so not exactly a common occurrence. The issue with Prices is a little more complex, and I had noted some oddities on occasion, that I noted I must look into but could be complex, and because I develop on one PC and use on another, it kept slipping off the radar, until I discovered the Summary issue, which used similar code and was usually in the same block of code. Of course, with the Summary issue I knew where to start checking, and so all was revealed quite quickly, including the obviously related Prices code. As far as the code goes, all is fixed now, but it may have resulted in incorrect stored prices for an ebook, having mixed with the prices of another ebook ... that was one of the oddities I had seemingly seen on a rare occasion. I hope that last isn't true, and it was just a display issue ... if not, my apology ... and if you can't find me, I am hiding in a closet somewhere full of shame.

NOTE - This issue with Prices if they were recorded, should have only impacted stored history, not Current or Last or Start price ... unsure about Low and High, but I suspect unlikely. So at the end of the day, just a faulty record in places if it did happen. Still, you have my sincere apology if it did.

(v8.1)
BUGFIX for retrieving Summary and Prices from file.
Bugfix for Publication Date.
Bugfix for Kobo Price comparison decimal issue.

NOTE - Had to re-upload after fixing another bug which crashed the program ... damn bugs. When adding the new Kobo code I overlooked the impact of the leeway variable being added to current price and making the decimal disappear, which another section of code didn't take too kindly too ... my fault again for not doing an array check for the existence of the second element ... oops.

Edited January 23, 2021 by TheSaint

[TheSaint]

KindEbook Wishlist has been updated to v8.2. See Post #2 on Page 1 for download.

MAJOR SPEED IMPROVEMENT

(v8.2)
Significant speed increase for loading the List in non sort mode.

Spoiler

NOTES - Been meaning to have a look and see if I could speed list loading up a bit (yet again), and finally got around to it. For most users with not many list items on an average PC, you would not have been impacted as I was, with 3884 INI section entries on my bought list (with around 16 or more keys each). Before I made changes today, that list took over 14 minutes to load. I culled over a minute with my first lot of changes, but my second lot of changes reduced loading time to just 27 seconds ... a whopping improvement. At this stage I have only done the first lot of improvements to the alternate sort mode loading option, but I will look at going further with that soon, though it is a lot more complex.

Generally my usage is to have sort set for my normal user and unsorted for my bought user. My normal user only has 279 entries currently, so loads in about 13 seconds. I rarely look at my bought user, so have tolerated its ever more slow loading time until now, for the few times I did check it. I have the bought user as unsorted so that entries are in chronological order of when bought.

So what did I change?
Well a good while ago I needed to modify my code to deal with too many sections in my INI file. So I changed to using FileRead which improved things greatly and overcame the INI reading limitation. However what I did wasn't as streamlined as it could have been, it was just a quickfix, that basically read every line but extracted the section names out of each appropriate line, then used that to get the keys and values via IniRead. So this time I split on line feed and "[" for my first change, which saved me over a minute with my bought user list. Then I split on each key= line which made the biggest speed improvement. So essentially no more INI reading to load the list in unsorted mode. So it's just FileRead to memory and get all required information from there.

I have pondered updating to SQL and even did some trials, but my script is over 12,000 lines and has way too many INI calls, so not an endeavor for the faint-hearted like me who just wants a simple life.

 

 

Edited January 24, 2021 by TheSaint

[TheSaint]

KindEbook Wishlist has been updated to v8.3. See Post #2 on Page 1 for download.

MAJOR SPEED IMPROVEMENT

(v8.3)
Significant speed increase for loading the List in sort mode, along with a small correction to code changes made in v8.2.
Clicking on HIGH column header now does nothing, while clicking the LAST column header just jumps and selects last list entry (what HIGH previously did because there was no LAST column at the time that feature was added, and unknowingly it had been overlooked until now).
Clicking on START column header jumps to first entry on the list.
I also added some timed dialogs to give better clarity about column sorting.

Spoiler

NOTES
Today I decided to tackle the somewhat more complex speed improvements for sorting on load of the list, and it wasn't as complex as I had thought. I've now reduced the time for my normal user list loading from around 13 seconds to 2.4 seconds, or because I also have Exchange Rate happening for me, 4.3 seconds (depending on how quick you respond to the two or three dialogs that appear ... User, Exchange Rate, Kobo price advice). It seems my small correction may have increased the speed again, as my bought user only took 18.8 seconds today, to load unsorted. Sorting on Author column for my bought user currently now takes 3.5 minutes for 3885 entries, whereas previously it was well over 14 minutes.

I had to add a line feed back in after splitting on that and the "[" ... that is the small correction mentioned above. It had minor impact without, just effecting where a required key came last in a section, which was really only relevant and important when sorting.

 

Edited January 24, 2021 by TheSaint