AutoHotkey Web Scraping

Alternative Web Page HTML Download Techniques (AutoHotkey Tip)

Jack Dunning AutoHotkey, Web browser, Web Pages , ,

When One Method for Downloading HTML Code Breaks, Try the Alternative AutoHotkey Command

After noticing that, although I could quickly get the latitude and longitude for any location with a Google search in a browser, when I attempted to download that page using the GetWebPage() function code taken from the AutoHotkey documentation (shown below in the first script), Google stopped me. The Google server denied the download attempt of the coordinates for San Diego with the following statement:

403. That’s an error.

Your client does not have permission to get URL /search?q=latitude+longitude+san+diego+decimal&rlz=1C1GEWG_enUS953US953 from this server.

Thwarted by Google again (see my “Switched IPFind.ahk to OpenStreetMap.org for Reliable AutoHotkey GUI Map Embedding” blog), I wanted to find an alternative source for the same information.

I searched for an unblocked Web page providing the latitude and longitude. I didn’t have to look very far. (The site name appears in the AutoHotkey snippet below.) I wrote the following test code for proof of function:

Continue reading →

Use ActiveX Control to Embed World Maps in AutoHotkey GUI

Jack Dunning ActiveX, Arrays, AutoHotkey, GUI (Graphical User Interface) Pop-ups, Link controls, Loops, Regular Expressions (RegEx) , ,

By Directly Loading a Map from OpenStreetMap.org into Your AutoHotkey Graphical User Interface (GUI) Pop-up Window, You Can Add Interactive Geographic Locations to All Your Apps

I have some good news and some bad news about using AutoHotkey tools to directly access Web data through the Internet. First the bad news. Since the AutoHotkey tools for downloading and reading Web pages use Internet Explore (built into Windows but no longer supported by Microsoft), Web providers can effectively block access by identifying that user browser. For the good news, you rarely need to use those sites blocking simple little personal apps such as my IPFind.ahk script. So many other sites support location data for IP addresses that I don’t have a problem keeping the script up and running.

For a quick glance at the geographic location of an IP address, insert an OpenStreetMap.org Web window into an AutoHotkey GUI. Hold the mouse cursor over a map and scroll in or out to zoom in or out.

Previously, I had repaired other issues caused by changes in the source Web page and converted the IPFind.ahk script to use a GUI window rather than a MsgBox command. This upgrade facilitated adding links to the app, see “Adding Web Links to the AutoHotkey IPFind.ahk Script,” as well as making the current insertion of interactive maps using the ActiveX GUI control possible. I fixed the IPFind.ahk script problems by switching to another source Web page and added an interactive map from OpenStreetMap.com.

Continue reading →

Adding Web Links to the AutoHotkey IPFind.ahk Script

Jack Dunning Arrays, AutoHotkey, GUI (Graphical User Interface) Pop-ups, Link controls, Loops, Regular Expressions (RegEx), Text Formatting, Web Pages , , , , ,

While Fixing the IPFind.ahk Script for Listing the Geographic Location of an IP Address, I Added Links for the IP Identification Site and OpenStreetMap

Occasionally, Web page scraping apps fail (or display strange results) due to changes in source page data formats. It usually only takes a few minutes to review the code and make the necessary RegEx adjustments to restore acceptable results. This time while repairing the IPFind.ahk script, I noticed that the Web page source code also offered IP longitude and latitude. I thought, “Why not add a map link to the display window for anyone curious about its geographic position?” The IP site (which I also added as a link) includes a map, but I wanted one with greater detail.

An IP address site can provide a great deal of information—including approximate longitude and latitude.

Note: I recently discussed the Link GUI control in “Turn Web Addresses into Hotlinks for the AHK File Peek Window.”

Continue reading →

Fixing AutoHotkey Web Lookup Scripts

Jack Dunning AutoHotkey, Cool Scripts, Menus, Regular Expressions (RegEx), Web Pages , , ,

If a Web Page Changes Format, the Data-Extracting Regular Expressions (RegEx) May Need Updating—Fixing the SynonymLookup.ahk Script

When writing a blog, I tend to use certain words over and over again. While rereading early versions, these redundant words jump out at me. Not only do they point out my limited vocabulary, but the repetitions tend to render my blogs a little more starchy and boring. That’s why I often resort to my always-loaded SynonymLookup.ahk script. This app saves time while making me look a little smarter.

The current version of SynonymLookup.ahk script lists more possibilities and marks antonyms (most of the time) with a caution sign (). (Click image for expanded view.)

After I discover a duplicated word, I highlight it, then hit the Ctrl+Alt+L Hotkey combination. A menu of possible replacements pops up. I click on the one that best fits my intent and the new term immediately displaces the original text. I habitually use this script.

When the SynonymLookup.ahk Script Breaks

Over the life of the script, I’ve encountered the menu shown at right a couple of times. This menu pops up whenever the script downloads and scans the source code 10 times without getting a RegEx hit—usually the result of code changes made by the source page Webmaster.

Continue reading →

AutoHotkey Tips of the Week: The ComObjCreate() Function for Web Page Downloads, E-Mail, and Text Audio

Jack Dunning AutoHotkey, Children's Educational Apps, ComObjCreate(), E-mail, Intermediate Level, Internet connection, Programming, Regular Expressions (RegEx), Windows scripting, Windows Speech , , , , , ,

While AutoHotkey Directly Supports Most Windows Features, the Flexibility of the ComObjCreate() Function Adds More Useful Capabilities—Especially for Capturing Web Data, Sending E-mail, and Reading Text Out-Loud

A number of my scripts use the ComObjCreate() function in various forms. Most of them I copied from the AutoHotkey Forums and modified for my own purposes. In this blog, I highlight the ComObjCreate() applications I use most, then offer a list of other forms of the function you may find useful.

How I Use ComObjCreate()

Synonym Page
The SynonymLookup.ahk script pulls replacement terms for the highlighted word “Page” from the Web.

While AutoHotkey supports many of these features in one form or another, directly accessing the COM (Component Object Model) might provide a solution you can get by no other method. I use the ComObjCreate() function in three ways:

  1. Collect data from Web pages (ComObjCreate(“WinHttp.WinHttpRequest.5.1”)).
  2. Send e-mail directly from an AutoHotkey script (ComObjCreate(“CDO.Message”))—no mail program required.
  3. Use the computer voice to read text (ComObjCreate(“SAPI.SpVoice”)).

While I haven’t found much additional information about the ComObjCreate() function posted on the new AutoHotkey forum, the old forum contains a useful COM Object reference list. You don’t need to know how they work—just how to use them. Continue reading →