In the early days of Asterisk®, it was not unlike the Wild West. Extension passwords of 1234 were pretty safe, and "anything goes" was the only rule when it came to scraping other folks’ web sites for data that you needed. And the master scraper of them all, Google, even had a phone: function (until they didn’t) that let you look up almost anyone’s name or telephone number. Ma Bell pretty much had a monopoly on CallerID Name (CNAM) data for phone numbers which is probably the reason the Google feature suddenly disappeared. Since the telecom industry universally discarded CNAM data when routing calls to destinations, it was up to the DID provider to look up the associated names out of their proprietary databases before connecting the call. Ma Bell called it a feature. It was more like a revenue generator.

For VoIP users, there were no CNAM services to speak of. Thus was born our CallerID Trifecta and later CallerID Superfecta projects more than a decade ago. Superfecta was later elevated to the stratosphere by some of the early FreePBX® pioneers including Lorne Gaetz and Andrew Nagy. The real beauty of CallerID Superfecta was the ability to search a cache for a CNAM match before scouring and scraping the web, a process which took a lot more time.

Much has changed in the last 10 years. One of the early lessons in CNAM lookups obtained by scraping web sites was that most of the site owners didn’t appreciate their newfound popularity. As a result, they continually modified their sites to discourage scraping, and this in turn regularly broke CallerID Superfecta. The other major change is that there now are multiple CNAM services for VoIP users. Almost all VoIP providers also include a CNAM lookup option, and there are dirt-cheap standalone services such as OpenCNAM and BulkCNAM.

In introducing CallerID Superfecta to the VitalPBX community, we wanted to take advantage of what we’ve learned as well as the new CNAM landscape. There still are limitations. All of the CNAM lookup services don’t mind charging you to return a CNAM result of Los Angeles, CA which you could have deciphered by simply examining the area code of the incoming call. And, if your girlfriend calls you twenty times a day, the CNAM services are more than happy to charge you for the same CNAM dip twenty times. And at least one VoIP provider charges for CNAM lookups whether you want them or not. That adds another half a cent to the cost of every incoming call. Most providers charge between $.002 (new BulkCNAM rate) and $.005 per dip1 so the charges can add up particularly with a lot of calls of short duration.

We previously mentioned using VitalPBX’s native OpenCNAM functionality to add CallerID Names to incoming calls. But there are three possible concerns with this approach. First, there’s the frequently calling girlfriend issue. Depending upon how much you like your girlfriend, that may or may not be acceptable. A second issue is the CNAM dips that only return the city and state of the call. For frequent callers, you obviously would like actual names associated with these callers. And a third issue is the absence of CNAM data in your CDR logs and on your SIP phone for outbound calls.

In considering a modern CallerID Superfecta design, we’ve concluded that you really only need three components. First, you need to examine whether an actual name was passed with the incoming call. Second, you need to check whether your local database already has a CNAM entry for the caller. And, third, if those two sources don’t bear fruit, you need a lookup from a stand-alone CNAM provider. We believe the prudent design for the local component is to consolidate all available CNAM information into a single local CNAM database with import tools from other servers for apps such as the Asterisk Phonebook, AsteriDex, and Google Contacts. The local Asterisk Phonebook included in AstDB’s SQLite3 database is perfectly suited to serve as a local CNAM repository.

Legal Disclaimer: Most CNAM providers have restrictions regarding caching of CNAM data. The courts consistently have ruled that phonebook data is not copyrightable. And every PBX caches CNAM data. After all, that’s what CDR logs are all about. Consult with your own attorney if you have concerns, or simply stop reading here. 🙂

Now that VitalPBX has included unlimited Custom Contexts at no cost, our tasks today are easy. The plan of attack goes like this. When an incoming call hits your PBX, we’ll check whether a CallerID name accompanied the call. Next, we’ll check the Asterisk Phonebook to see if we have a name associated with that phone number. If there’s no CNAM entry after checking both sources, then we’ll perform an OpenCNAM lookup using your credentials. For outgoing calls, we’ll check the Asterisk Phonebook for a CNAM match on the number being called and, if there is no match, we’ll perform an OpenCNAM lookup using your credentials. You have the option of using either, both, or neither of these CNAM scripts.

Before we get started, it’s worth noting that finding a "hook" for outbound calls was not a simple task with VitalPBX. The trick was to find a subroutine that could be replaced without damaging the native functionality of the PBX. We finally found a spot, but we were limited to just two lines of code. The solution was to use one of those lines to execute a separate subroutine which had no restrictions on size. So let’s get started.

As with FreePBX, a config file is used to manage custom contexts. With VitalPBX, extensions__80-custom.conf in /etc/asterisk/ombutel is the file you need. If it doesn’t exist, simply create it. This file has two parts which again are similar to the FreePBX design. The [cos-all-custom](+) context is used for extension code in exactly the same way that the [from-internal-custom] context is used in FreePBX. In addition to extension dialplan code, you also can embed additional contexts just as you could with FreePBX. If you’ve installed Incredible PBX for VitalPBX, then you already have a number of extension components in the [cos-all-custom](+) context. So we’ll be adding some additional contexts at the bottom of the file. If you don’t already have an extensions__80-custom.conf file, create it and add [cos-all-custom](+) as the first line in the file. All of today’s code is licensed for your use pursuant to the terms of the GPLv2 license available here.

Adding CallerID Superfecta for Incoming Calls

Log into your server as root and navigate to /etc/asterisk/ombutel. Edit extensions__80-custom.conf and cut-and-paste the following code to the bottom of the file. Be sure to insert your actual OpenCNAM credentials in the SID and TOKEN entries near the top of the code replacing xxxx and yyyy.

;# // BEGIN CallerID Superfecta
[superfecta]
exten => s,1,NoOp(Executing CID Superfecta)

; insert OpenCNAM credentials below
  same => n,Set(SID=xxxx)
  same => n,Set(TOKEN=yyyy)

  same => n,NoOp(CID Number: ${CALL_SOURCE})
  same => n,Set(CID_NAME=${CALLERID(name)})
  same => n,NoOp(AstDB lookup underway)
  same => n,Set(CID_NAME2=${DB(cidname/${CALL_SOURCE})})
  same => n,GotoIf($["foo${CID_NAME2}" = "foo"]?runOpenCNAM:skipOpenCNAM)
  same => n(runOpenCNAM),NoOp(OpenCNAM lookup underway)
  same => n,Set(DB(cidname/${CALL_SOURCE})=${CID_NAME})
  same => n,Set(CID_NAME3=${SHELL(curl https://${SID}:${TOKEN}@api.opencnam.com/v2/phone/${CALL_SOURCE}?format=pbx)})
; for BulkCNAM, comment out line above and uncomment line below after inserting your APIkey
; same => n,Set(CID_NAME3=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CALL_SOURCE}")})
  same => n,GotoIf($["foo${CID_NAME3}" = "foo"]?skipBOTH
  same => n,Set(DB(cidname/${CALL_SOURCE})=${CID_NAME3})
  same => n,Set(CALLERID(name)=${CID_NAME3})
  same => n,Goto(skipBOTH)
  same => n(skipOpenCNAM),Set(CALLERID(name)=${CID_NAME2})
  same => n(skipBOTH),Set(ICALL=yes)
  same => n,Answer

;  same => n,Wait(2)
;; requires addition of nv-GenericWelcome.ulaw in /var/lib/asterisk/sounds/custom
;  same => n,Background(custom/nv-GenericWelcome&silence/1&continue-in-english&press-7,nm)
;  same => n,WaitExten(.)
;; exten => 7,1,Goto(cos-all,701,1)
;; exten => 7,1,Goto(cos-all-trunk,8436060555,1)
;; exten => 7,1,Goto(newcontext,s,1)
;  exten => 7,1,Goto(incrediblepbx,s,1)
;  same => n,Return()
; exten => t,1,Hangup
;  same => n,Return()
; exten => i,1,Hangup
;  same => n,Return()

; same => n,Goto(cos-all,701,1)
; same => n,Goto(cos-all-trunk,8436060555,1)
; same => n,Goto(newcontext,s,1)
; same => n,ExecIf($["${DID}" = "8436060555"]?Goto(incrediblepbx,s,1))
  same => n,Goto(incrediblepbx,s,1)
  same => n(return),Return()
;# // END CallerID Superfecta

Replacing OpenCNAM with BulkCNAM for Queries

With the recent price drop, you may prefer to use BulkVS.com for your CNAM queries. You’ll need to obtain your APIkey from their portal in API:API Credentials. Then locate the Set(CID_NAME3… line in the script above and replace it with the following line using your actual APIkey:

same => n,Set(CID_NAME3=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CALL_SOURCE}")})

Choosing a Destination for Incoming Calls

We want to spend a minute on the last several lines of the Superfecta dialplan code:

; same => n,Goto(cos-all,701,1)
; same => n,Goto(cos-all-trunk,8436060555,1)
; same => n,Goto(newcontext,s,1)
  same => n,ExecIf($["${DID}" = "8436060555"]?Goto(cos-all,701,1))
  same => n,Goto(incrediblepbx,s,1)

 
These five lines demonstrate how to redirect incoming calls to five different destinations: an extension, an external phone number, another custom context, a DID-specific routing, or the Incredible PBX Demo IVR. As shown, the calls would be routed to the default Demo IVR unless the DID matches 8436060555 in which case the call would be routed to extension 701. You can modify the destination scenario simply by uncommenting a different line in the dialplan examples. Keep in mind that a Goto command permanently redirects calls to that destination outside the Custom Context so the first uncommented Goto line will always take precedence. Also note that you can have multiple ExecIf lines preceding a Goto line to route calls to different destinations depending upon the DID of the incoming call. Stated another way, multiple ExecIf commands can be followed by a Goto command that serves as the default destination for any non-matching DIDs. You could also redirect by CallerID number.

Blocking Telemarketers with CallerID Superfecta

As election season heats up in the United States, be advised that your favorite legislators have exempted themselves from the robocall rules so dinnertime will get more interesting in coming months. One simple solution is to prompt the caller to enter a number before actually connecting the call. This kills most robocalls. We’ve included the code to do this in CallerID Superfecta. Just uncomment the code so that it looks like the following after choosing a call destination as documented in the previous section:

  same => n,Wait(2)
; requires addition of nv-GenericWelcome.ulaw in /var/lib/asterisk/sounds/custom
  same => n,Background(custom/nv-GenericWelcome&silence/1&continue-in-english&press-7,nm)
  same => n,WaitExten(.)
; exten => 7,1,Goto(cos-all,701,1)
; exten => 7,1,Goto(cos-all-trunk,8436060555,1)
; exten => 7,1,Goto(newcontext,s,1)
  exten => 7,1,Goto(incrediblepbx,s,1)
  same => n,Return()
 exten => t,1,Hangup
  same => n,Return()
 exten => i,1,Hangup
  same => n,Return()

 
Before this will work, you’ll need to download the nv-GenericWelcome.ulaw recording:

cd /var/lib/asterisk/sounds/custom
wget http://incrediblepbx.com/nv-GenericWelcome.ulaw

Adding CallerID Superfecta for Outgoing Calls

If CNAM data for outgoing calls is also desired for your CDR logs, here’s the dialplan code to cut-and-paste into the bottom of the Custom Contexts file. Insert your OpenCNAM credentials in the code replacing xxxx and yyyy just as you did in the previous section:

[cos-all-custom]
exten => _.,1,NoOp(*** Custom Outbound CNAM Code ***))
    same => n,GoSub(cnam-out,s,1)

[cnam-out]
exten => s,1,NoOp(*** Outbound CNAM context for Incredible PBX ***)
  same => n,Set(SID=xxxx)
  same => n,Set(TOKEN=yyyy)
  same => n,NoOp(Calling: ${EXTENSION})
  same => n,Set(CID_NUM=${EXTENSION})
  same => n,NoOp(AstDB lookup underway)
  same => n,Set(CID_NAME=${DB(cidname/${CID_NUM})})
  same => n,GotoIf($["foo${CID_NAME}" = "foo"]?runOpenCNAM:skipOpenCNAM)
  same => n(runOpenCNAM),NoOp(OpenCNAM lookup underway)
  same => n,Set(CID_NAME=${SHELL(curl https://${SID}:${TOKEN}@api.opencnam.com/v2/phone/${CID_NUM}?format=pbx)})
; for BulkCNAM, comment out line above and uncomment line below after inserting your APIkey
;  same => n,Set(CID_NAME=${SHELL(curl -X GET "https://cnam.bulkCNAM.com/?id=APIkey&did=${CID_NUM}")})
  same => n,Set(DB(cidname/${CID_NUM})=${CID_NAME})
  same => n(skipOpenCNAM),Set(CDR(customer_code)=${CID_NAME})
  same => n,Set(ICALL=yes)
  same => n,NoOp(*** Outbound CNAM context ends here ***)
  same => n,Return()

Activating CallerID Superfecta with VitalPBX

As with any dialplan changes, you first must reload the dialplan: asterisk -rx "dialplan reload". Next, we need to create a Custom Context within the VitalPBX GUI and point any desired Inbound Routes to that custom context. Using a web browser, log in to the VitalPBX GUI as admin. Navigate to PBX:Applications:Custom Contexts and create a new custom context for CallerID Superfecta:

Description: CID Superfecta
Context: superfecta
Extension: s
Priority: 1
Destination: Terminate Call -> Hangup

Next, identify the Trunks that you want to use with CallerID Superfecta and create an Inbound Route for those trunks pointing to CallerID Superfecta as the Destination for those calls.

If you want different destinations within CallerID Superfecta context for different trunks, then you’ll need to use ExecIf commands as shown in the dialplan to identify each trunk and to route the calls to the desired locations. Note that you can have multiple ExecIf commands followed by an uncommented default destination for any incoming calls that do not have a DID match.

Importing Asterisk Phonebook Data from Another PBX

To supplement your local Asterisk Phonebook which will serve as the local CNAM repository for VitalPBX, you can easily import data from other sources. And, as those sources change, you can export the entire database again and repeat the import procedure without worrying about duplicating data in your local Asterisk Phonebook. Imported data always overwrites existing entries which have a match on the phone number. To import an Asterisk Phonebook from another PBX, here are the steps to run on that PBX to create the import script which should be copied and run on the VitalPBX platform: import-asterisk-phonebook.sh

CAUTION: Carefully review the generated import script for anomalies before running it.

cd /var/lib/asterisk
sqlite3 astdb.sqlite3 "select key,value from astdb \
 where key LIKE '%cidname/%'" > /root/import.sh
cd /root
sed -i 's|/cidname|asterisk -rx "database put cidname "|' import.sh
sed -i 's| "/| |' import.sh
sed -i 's|\|| "|' import.sh
sed -i 's|$|"|' import.sh
sed -i "s|\"d|'d|" import.sh
sed -i "s|$|'|" import.sh
chmod +x import.sh
mv import.sh import-asterisk-phonebook.sh

Importing AsteriDex Data from Another PBX

Here are the steps to export existing AsteriDex data from another Incredible PBX platform. Once the steps have been completed, copy import-asteridex.sh to VitalPBX, make the script executable, and run it.

mysql -u root -ppassw0rd asteridex -N -s -e 'select "*",`out`,`name` from user1' | \
 sed 's|\t|, |g' > tmp.txt
sed -i "s|*,|asterisk -rx 'database put|" tmp.txt
sed -i 's|, | "|' tmp.txt
sed -i 's|$|"|' tmp.txt
sed -i "s|$|'|" tmp.txt
mv tmp.txt import-asteridex.sh

Importing Google Contacts into Asterisk Phonebook

Here are the steps to import existing Google Contacts into the VitalPBX Asterisk Phonebook:

  1. From within Google Contacts, export your contacts into a vcard format file: contacts.vcf
  2. Copy the contacts.vcf file to the /root folder of your PBX
  3. wget http://incrediblepbx.com/googlecontacts-importer.tar.gz
  4. tar zxvf googlecontacts-importer.tar.gz
  5. rm -f googlecontacts-importer.tar.gz
  6. Run the main script: /root/vcard-import.sh
  7. Edit the shell script that is generated: nano -w import-google-contacts.sh
  8. Make any changes to clean up the entries from Google Contacts
  9. chmod +x /root/import-google-contacts.sh
  10. Run the script to import your Google Contacts: /root/import-google-contacts.sh

Managing the Asterisk Phonebook with VitalPBX

To add or replace entries in the Asterisk Phonebook from the Linux CLI, here’s the syntax:

asterisk -rx 'database put cidname 8005551212 "Directory Assistance"'

To remove an entry from the Asterisk Phonebook, the syntax looks like this:

asterisk -rx "database del cidname 8005551212"

To display all entries in your Asterisk Phonebook, issue the following command:

asterisk -rx "database show cidname"

 
Originally published: Monday, November 4, 2019



Need help with Asterisk? Join our new MeWe Support Site.


 

Special Thanks to Our Generous Sponsors

FULL DISCLOSURE: RentPBX, Amazon, Skyetel, Vitelity, DigitalOcean, Vultr, Digium, 3CX, Sangoma, TelecomsXchange and VitalPBX have provided financial support to Nerd Vittles and our open source projects through advertising, referral revenue, and/or merchandise. We’ve chosen these providers not the other way around. Our decisions are based upon their corporate reputation and the quality of their offerings and pricing. Our recommendations regarding technology are reached without regard to financial compensation except in situations in which comparable products at comparable pricing are available from multiple sources. In this limited case, we support our sponsors because our sponsors support us.

BOGO Bonaza: Enjoy state-of-the-art VoIP service with a $10 credit and half-price SIP service on up to $500 of Skyetel trunking with free number porting when you fund your Skyetel account. No limits on number of simultaneous calls plus quadruple data center redundancy assures that you’ll never have a failed call. Tutorial and sign up details are here.

VitalPBX is perhaps the fastest-growing PBX offering based upon Asterisk with an installed presence in more than 100 countries worldwide. VitalPBX has generously offered to provide a customized White Label version of Incredible PBX that is specifically tailored for use with Incredible PBX and VitalPBX custom applications. Soon!
 

Special Thanks to Vitelity. Vitelity is now Voyant Communications and has halted new registrations for the time being. Our special thanks to Vitelity for their unwavering financial support over many years and to the many Nerd Vittles readers who continue to enjoy the benefits of their service offerings. We will keep everyone posted on further developments.
 


​​3CX is a software PBX that’s easy to install & manage. It includes integrated softphones, WebRTC conferencing and essential add-ons out of the box, at no additional cost. Try the free edition at www.3cx.com. Better yet, download the PIAF5 ISO powered by 3CX. Free version includes support for 8 simultaneous calls with a SIP trunk.

  • Run on Premise or in the Cloud, on Windows and now on Linux
  • Softphones for iOS, Android, Win & Mac
  • Easy install, backup & restore, version upgrades
  • Automatically configures IP Phones, SIP Trunks & Gateways

  • RentPBX, a long-time partner and supporter of PIAF project, is offering generous discounts for Nerd Vittles readers. For all of your Incredible PBX hosting needs, sign up at www.RentPBX.com and use code NOGOTCHAS to get the special pricing. The code will lower the price to $14.99/month, originally $24.99/month. It’s less than 50¢/day.


    Print Friendly, PDF & Email

    Be Sociable, Share!

    1. BulkCNAM previously charged $.005 per CNAM dip but recently sent out an email indicating they were dropping the rate to $.002. Our CNAM charges continued to be billed at $.005 with the daily charges being rounded up to the whole penny. So… you may want to check your CNAM charges for a bit to verify that you are being billed at the new rate. If not, open a ticket and they will correct it. []

    Leave a comment

    Your email address will not be published. Required fields are marked *

    *