If the Source Code for a Web Page Changes, You May Need to Rewrite Your Web Data-Driven Script, Plus More AutoHotkey Tips

Because I do a great deal of writing, I’ve fallen in love with my SynonymLookup.ahk script which pops up a menu of alternative words for instant replacement in my documents. It’s pretty cool—even if I do say so myself. However, the other day, it ceased working.

After highlighting a redundant word, I initiated the Hotkey combination searching for an equivalent term. Nothing happened! No matter how much time passed, the script displayed nothing.

After investigating, I discovered that (exactly as I had contemplated in the first SynonymLookup.ahk blog) Thesaurus.com had changed the formatting of the target page source code causing my Regular Expression (RegEx) to fail. The manufactured GoTo loop I used to increase the reliability of the Web download created an infinite loop.

I needed to adapt.

I don’t presume that the changes made by Thesaurus.com had anything to do with my little script, but I would understand why a commercial Web site might not want a quick and easy Windows app to piggyback off their pages. Ultimately, SynonymLookup.ahk acts as a specialized browser—although not displaying any ads. Rather, it bypasses the tedious process of loading the Web page into your browser, scrolling while hunting for a useful alternative term, copying that word, then, after jumping back to your original editing window, the pasting of the synonym back into your document. This type of AutoHotkey script defeats the basic business model for advertising-supported sites.

As sympathetic as I am to the plight of the Web advertising community, I decided to fix my app. It took me about five minutes to change the RegEx to display the synonyms again, but I wanted to add a few other improvements—one of which gives more credit directly to Thesaurus.com:

1. The source code format change at Thesaurus.com caused an infinite loop in the script. I added a trap to the artificial GoTo loop which breaks the iterations and provides the user with a link for updating the script.
2. After rewriting the RegEx, the new synonym values need symbol replacement for special characters (e.g. tête-à-tête) which now appear in the variable.
3. I give credit to the data source by offering users direct access to Thesaurus.com through a highlighted menu link (). (I previously offered the link through the bold menu heading, but it did not include the site name.)
4. The menu now includes antonyms which I marked with a caution sign () to prevent confusion.

I like the new version of SynonymLookup.ahk even better.

I suppose that in the future Thesaurus.com might make other changes which could totally stifle my efforts to maintain a quick synonym lookup and replacement tool, but, for now, I’ll keep using this script. Eventually, if it appears impossible to continue with Thesaurus.com as a data source, I’ll likely switch to one of the other synonym sites. (I checked out a couple of them and saw features in the code format which could potentially produce a truly more useful SynonymLookup.ahk.)

Checking for Bad Data from Web Page

The SynonymLookout.ahk failure pointed out a major problem with the artificial GoTo loop I implemented to improve Web download reliability. If a non-operational RegEx finds no synonyms, it blunders into an infinite loop while offering no visible signs of the trap. For the average user, it merely looks like the app no longer works.

I implemented a counter to limit the number of GoTo loops to 10. At that point, the script tells the user, “App Broken!” and adds a menu option to allow checking for an update to the script. (See image above.)

Before the GoTo loop Label, initiate the variable Try_Count:

Try_Count := 0

After the primary internal synonym Loop, but before the check for the GoTo loop, increment the variable by 1 (Try_Count++) and add the trap for exiting the GoTo loop by adding one menu item to the Synonym menu:

Try_Count++   ; Count number of GoTo loops
If (Try_Count = 10)
{
  Menu, Synonym, add,App Broken! Check for update?, LoadAppPage
  Menu, Synonym, Icon, App Broken! Check for update?
         , %A_Windir%\system32\SHELL32.dll, 78,
}

This adds the menu item for accessing the latest SynonymLookup.ahk the ComputorEdge Free AutoHotkey Scripts and Apps page using the Label subroutine LoadAppPage:

LoadAppPage:
  Run, http://www.computoredge.com/AutoHotkey/
         Free_AutoHotkey_Scripts_and_Apps_for
        _Learning_and_Generating_Ideas.html#SynonymLookup
Return

(Note: The Run command statement above wraps for display purposes only. It does not represent a valid line continuation. To work it must appear in the script as one continuous line.)

Fixing Special Characters

Since the new RegEx uses the embedded links found in the source code, we find many special characters represented by codes (e.g. %C3%A9 ⇒ é). I added the StrReplace() function to replace these codes with the correct symbol:

Synonym1 := StrReplace(Synonym1,"%27","'")
Synonym1 := StrReplace(Synonym1,"%C3%A9","é")
Synonym1 := StrReplace(Synonym1,"%C3%AF","ï")
Synonym1 := StrReplace(Synonym1,"%C3%AA","ê")
Synonym1 := StrReplace(Synonym1,"%C3%A0","à")

As I continue using the script, I expect to encounter more of these inconsistencies.  I plan to insert more correction statements and update the script on the download site as I find these disparities.

Menu Links to Thesaurus.com

To give clear attribution to Thesaurus.com as the source of data, I added two items to the end of the Synonym menu:

Menu, Synonym, add, Visit, LoadPage, +Radio
Menu, Synonym, check, Visit
Menu, Synonym, Icon, Visit, %A_Windir%\system32\SHELL32.dll, 44,
Menu, Synonym, add, Thesaurus.com, LoadPage
Menu, Synonym, Icon, Thesaurus.com, %A_Windir%\system32\SHELL32.dll, 44,
Menu, Synonym, Show

These links use a star icon () to highlight the entries. Both items link directly to Thesaurus.com.

Mark the Antonyms

The synonym list now includes antonyms (opposites) when applicable. If accidentally selected, these words could cause confusion. I’ve added a caution icon () to each antonym by identifying the positions where they start and end in the Web page source code:

AntonymPos := RegExMatch(SynPage,"Antonyms <span class") 
AntonymStop := RegExMatch(SynPage,"More words related to <strong>")

Then, whenever found between those two points (and not 0), the word gets marked as an antonym in the menu:

If (AntonymPos != 0) and (AntonymPos < SynPos) 
                     and (SynPos < AntonymStop)
    Menu, Synonym, Icon, %Synonym1%, %A_Windir%\system32\SHELL32.dll, 78

This makes it easy to spot the contrary terms and ignore them—or deliberately choose a word when looking for the opposite of the highlighted input.

*          *          *

My list of projects seems to grow but I have not forgotten them. I owe readers:

1. Techniques for determining when a Web page completes loading into a Web page.
2. The next step in “Input Command Creates Temporary Hotstrings from Data Table (AutoHotkey INI File Technique)” which add more access to Latin legal terms.
3. Continued work on the new AutoHotkey Quick Reference script.

Don’t worry. I’ll keep plugging away.

This post was proofread by Grammarly
(Any other mistakes are all mine.)

(Full disclosure: If you sign up for a free Grammarly account, I get 20¢. I use the spelling/grammar checking service all the time, but, then again, I write a lot more than most people. I recommend Grammarly because it works and it’s free.)