About tnolen

This author has not yet filled in any details.
So far tnolen has created 267 blog entries.

Web Server Problem

September 27, 2021 – 9:30am – We are experiencing an issue with a shared web server which is affecting some customers.   We are currently investigating the issue as it is taking longer to recover than we initially thought.  We’ll be posting updates here as we get them.

Update 10:03am – Services have been restored.  A combination of DOS (denial of service) attack and hardware failure was responsible.  We are working on a hardware fix.

 

If you have any questions or concerns about this, please email us at support@internetpro.net, text or call 2565476817, or use the chat box in the lower right of your screen.

 

 

2021-09-27T10:36:25-05:00September 27, 2021|News and Updates|Comments Off on Web Server Problem

Phone Issues

September 25, 2021  3:30pm  – We are getting reports of our customers not being able to receive calls from Verizon.  Right now, only Verizon has been tested.   We’re working on other tests to see if we can narrow down the issue.

We will post updates here as we have them.

Update 3:40pm – We have confirmed that this is a Verizon issue.   It is affecting Verizon wireless customers and customers of cellular services which use their networks such as Spectrum and Straight Talk.   Downdetector is also reporting Verizon issues here: https://downdetector.com/status/verizon/.

Update  3:55pm – It looks like the worst has passed.   Downdetector is showing that the reports of issues are starting to drop and we have had reports of Verizon calls going through to our customers now.

 

September 27, 2021 – 10:00am – We are not sure the issues are related to this weekend’s events, but our carrier has posted a bulletin stating this:
“NOTICE: Currently a large underlying carrier is experiencing intermittent service disruption. This outage is causing inbound calls of some numbers to not be delivered to SIPTRUNK. We apologize for any service interruption that these issues may cause. “

Update 11:30am – We have discovered that DTMF tones are no longer working on digital receptionists.  DTMF tones are the tone made by your phone when you dial on the keypad and they are used to choose options on the menus in phone systems (for instance, push 1 for sales, push 2 for support, etc.).   Because of this, people are not able to make choices on those menus.  Some customers have their systems set to hang up if no choice is made to weed out telemarketers.  Right now, those systems may be weeding out everyone.  If this seems to be the case on your system, please contact us and we can change the system so that “no choice made” goes somewhere. We have alerted our carrier to the issue in case they are unaware.

Update 11:48am – This seems to be a widespread issue as we have now received a report from a customer using AT&T as the carrier who is having similar issues.

Update 12:00pm from SIPTRUNK – “Our carrier is experiencing an intermittent issue with inbound and outbound calls. This includes audio & DTMF tones. But static is usually only introduced at analog equipment, so we suggest rebooting any physical devices you might have, including router, modems, firewalls, PBXs, etc. But we have multiple tickets open with the carrier. All teams are actively engaged at this time. Currently no ETA. We will provide an update as soon as one is available.”

Update 3:45pm – SIPTRUNK reports that the issue is still under investigation.

Update 6:35pm – Our carrier reports that the network behavior has returned to normal.  We are changing phone system programming that was changed because of the DTMF issues.   Please let us know if you are still experiencing abnormal phone system behavior.

 

Update September 28 8:45am – Although the carrier reported that things were back to normal, we are still receiving reports or issues (and experiencing them ourselves).   We have reported this and are awaiting news.

Update 4:30pm – Updates have been slow to come today.   Our carrier has given us a bit more information now though.  They are seeing about 1/7 the tickets today as yesterday, so things are improving.   The issue is not a misconfiguration, but rather is a DDOS attack directed at VOIP carriers.  The affected carriers have not released that it is a ransomware attack, but that happened recently to a large Canadian carrier last week and the rumor is that this is related and may be the same attackers.   Earlier, Verizon was the suspected source of the issue and even Verizon themselves thought they were the problem.  However, the underlying carrier currently being attacked is Bandwidth. Because of the interconnections between companies, it seemed to be a Verizon problem at first because they had the most end users affected.   Bandwidth is a large “behind the scenes” carrier.   You can check their Down Detector information here: https://downdetector.com/status/bandwidth/   and their own status page here: https://status.bandwidth.com/.   We expect them to post “everything normal” later today as off-hours voice traffic drops a lot and reduces the impact of the attacks, but we may see the same issues return tomorrow if they have not been able to mitigate the attacks.

Update September 29  8:45am – Currently, it looks like a repeat of yesterday, where daytime business traffic is starting to have an impact on performance of the network.  Downdetector is showing an increase of reported issues again: https://downdetector.com/status/bandwidth/.   Our carrier is working on diversifying their routes to other upstream carriers, but this involves a lot of manual porting of hundreds of thousands of numbers, so this will take some time.   Mitigation of the attack at the carrier level is improving, so we expect less problems as time goes on, but there *are* still issues happening.

Update 10:45am – Carrier has posted an issue this morning but are currently saying it only affects inbound calls from the Verizon network.  Of course, other providers using the Verizon network such as Spectrum Mobile, Straight Talk, etc. will also be affected.  Here is another recent news article with a little more information:  https://www.zdnet.com/article/bandwidth-ceo-confirms-outages-caused-by-ddos-attack/

Update 4:00pm – Siptrunk sent out a letter regarding the issue.  No real extra news, but some clarifications.   Here is an excerpt from the letter we received:

–BEGIN

As you may be aware, one of the US’s largest underlying carriers (ULC) has been under a Distributed Denial-of-Service (DDOS) attack this week. This attack is similar but larger in scope than the attack on VoIP.MS that started on September 16th.    Since this is a ULC we can easily route around them on outbound calls, but this is not the case for inbound calls. The ULCs serve as the “carrier of record” for the DIDs or telephone numbers used by our customers. The current design of the US telephone network means that all calls destined to a particular telephone number have to be handled by the carrier of record. The only way to change this is to port or move the number to a new carrier of record and for landlines, this process can take 5 to 7 days under normal circumstances.

SIPTRUNK is investigating all options to address intermittent inbound call issues caused by the DDOS on the ULC, but please know this is affecting large and small customers the likes of which including Microsoft Teams, Google voice, RingCentral, etc… to just name a few.

We have seen significant improvement on the ULC that is being attacked, and we have confidence they will get their network restored to 100% capacity as quickly as possible.

However, with that said SIPTRUNK has been working with other ULCs to arrange porting of DIDs to alternative ULCs. As you can imagine there are a number of carriers and customers attempting mass ports and this is causing congestion in the porting process. We will continue to work to have inbound calls delivered via alternative ULCs.

–END

September 30 – 7:15am – For the technically curious, here is a good video between a well-known MSP (Managed Service Provider) and a VOIP provider on Youtube discussing the issue.  It uses a lot of jargon, but may be interesting to some of you.  https://www.youtube.com/watch?v=CeCd9zW-R5o.  It is from yesterday, so when they say “yesterday”, they are talking about the 28th.  However, not much has changed.

Update 1:45pm – Upstream providers report much better stability of the network today.  There are still some users reporting issues, but the number and severity seems to be diminished.

Update 4:15pm – SIPTRUNK has released another statement.  Here are the relevant parts:
–BEGIN
All SIPTRUNK Network Services remain operational. We continue to monitor the underlying carrier (ULC) that has been impacted over this week. At this time, they remain under a DDoS attack however they are taking the necessary steps to limit service disruptions. We continue to see significant improvements as they fine-tune their DDOS mitigation efforts, and we have confidence they will get their network restored to 100% capacity as quickly as possible.

We continue to collaborate with other ULCs to organize emergency ports from the impaired carrier to their services. It has become clear that the DDoS attacks are impacting the ability of Providers to port numbers. Providers are experiencing significant porting delays and we are also aware of the potential disruptions that could occur when moving numbers. We will continue to test and try alternative solutions to move beyond the control of the attackers.

Please understand this is not just an attack on a ULC but an attack on the United States Infrastructure and it has visibility at all levels of the federal government.
–END

Update October 1 4:15pm – All ULC services have been stabilized and calls are processing normally.  We performed a scheduled port today and it went through without issue. We believe this issue has now been resolved.

 

If you have any questions, please text 256-547-6817 or email support@internetpro.net.

 

 

 

 

2021-10-01T16:21:30-05:00September 25, 2021|News and Updates|Comments Off on Phone Issues

Phishing Attempt *Updated-Again*

July 28, 2020 – We’ve had several customers inquire about a phishing attempt which is apparently hitting a lot of our customers.  An email comes in which looks very similar to the screenshot below:

 

Please note that we do not have any such system and we never go by the name “Internetpro”.   We do own the domain internetpro.net, but the name of the company is Network Solutions, Inc.   If you have responded to this email and given your username and password, please contact us to change it immediately, as  your credentials have been stolen.

 

Update 8/14/2021:  Another variation is making the rounds.  It look similar to this:

 

 

Update 8/28/2021:  Yet another variation…Looks like this:

And Another (9/14/2021):

2021-09-14T16:34:37-05:00July 28, 2021|News and Updates|Comments Off on Phishing Attempt *Updated-Again*

Widespread Internet Outage

July 22, 2021 11:30am – There is a widespread internet outage going on.  We have seen issues reaching Microsoft as well as other services using the Akamai CDN (Content Delivery Network).  Akamai’s network touches many public services, and they may not be the primary source of the problem.   We will be posting updates here as we find out more.

This is not an issue with our servers or services and is something affecting the internet as a whole.

Update 11:50am: We are hearing that Amazon Web Services (AWS) is also affected.  Many large sites run on their services, so this is affecting banks, news sites, and others.

Update 11:55am: Down detector https://downdetector.com/ is showing 45 major sites on their homepage.  The site has to be reporting a major issue to be featured there. They specifically list a problem at Akamai as the root issue.

Update 12:05pm:  Akamai reports that they have implemented a fix and services are returning to normal.  We have seen sites coming back online as well.  We expect full normalization within the hour.

 

 

If you have any specific questions about this outage, please feel free to contact us via chat below, or text messaging or calling 256-547-6817 or by emailing support@internetpro.net.

 

2021-07-22T12:35:26-05:00July 22, 2021|News and Updates|Comments Off on Widespread Internet Outage

Planned Maintenance

We will be conducting planned maintenance on Saturday, July 24th.  This maintenance will mostly affect VPS (virtual private server) customers.  Email systems and shared hosting services will experience little to no interruption.   We will be upgrading the infrastructure for some servers and storage systems.   This maintenance will take longer than usual, so we will be beginning a bit earlier than normal, starting at 10:00pm CDT.

July 24, 2021 9:35pm – Maintenance has started.  Most work is being done “behind the scenes” right now and will not be customer-facing.

Update 10:15pm – Customer facing sites have begun to be affected.

Update 10:35pm – Email services *are* going to be limited for a brief time…Starting now.

Update 10:59pm – Email services restored.

Update 11:30pm – Because of the maintenance, we had taken our proactive alerting system offline (too many alerts would be sent).  However, we missed a shared server having trouble and affecting customer sites.  We’re working on that.

Update 11:40pm – The aforementioned unplanned outage has been resolved.

Update 12:35am – As part of the maintenance continues, our phone system is now offline.

Update 3:00am – Phone system back online and VPS machines are returning.

Update 3:35am – Everything seems to be functioning normally now.  If you have any services which are not performing correctly, please let us know.

 

 

If you have any questions or concerns about this work, please email us at support@internetpro.net, call or text 256-547-6817, or reach out through the chat box below.

 

2021-07-25T03:37:29-05:00July 19, 2021|News and Updates|Comments Off on Planned Maintenance

Sparklight Outage

July 13, 2021 – Sparklight is experiencing a major outage which started about 10:45 CDT.   It has been ongoing now for about 2 hours, so we assume this will be an extended outage.   The issue has taken down their call center as well as their website at https://www.sparklight.com.   Customers in the Gadsden area are also affected.  We have not been able to reach anyone for an ETR (Estimated Time to Repair) and currently are just following reports on Twitter.  We will post updates as we get them here.

Update 3:30pm – Their call center has had service restored.

Update 4:50pm – We are getting reports that Sparklight is beginning to come back online, but we definitely have customers still affected.  No additional information has been posted on their Facebook page.

 

Some articles related to the outage:
KTVB.COM
Valley News Live
Sparklight Facebook

 

 

2021-07-13T16:57:08-05:00July 13, 2021|News and Updates|Comments Off on Sparklight Outage

Comcast Outage

  • June 28, 2021 4:00pm – Comcast seems to be experiencing an issue in south Gadsden.  We began receiving calls as early as 2:30pm, so this seems be an outage that is going to be lasting a bit.  Currently, the estimated time to repair (ETR) is 7:14pm central time.  That time is usually automatically generated for 4 hours from the time they declare that an area is having an outage, so we doubt it is accurate.   Unfortunately, that is all of the information that we currently have.   This outage seems to primarily be affecting the Country Club area.   We’ll post more information here as we get it.

Update 6:25pm – The ETR is still listed as 7:14 p.m. No additional updates are available.

Update 6:35pm – We are told that modems are beginning to come online right now.

Update 7:00pm – Confirmed that things are back online.

 

 

 

 

2021-06-28T19:28:35-05:00June 28, 2021|News and Updates|Comments Off on Comcast Outage

Web Server Issue

May 29, 2021 2:55pm – One of our shared web servers is having some issues and sites it is hosting are currently unreachable.  We are investigating the problem to determine if it is a hardware failure, denial of service attack, or some other problem. Sites on that server will just not respond, so if you currently are having trouble accessing your web site, this is most likely the problem.   We will post updates here as we find out more information.

Update – 3:35pm – Services have been restored.  Unfortunately, we haven’t yet been able to tell exactly what the root cause of the issue was, but we are still investigating it.  If we find anything particularly interesting, we’ll post an update here.

 

If you have any further concerns regarding this outage or if your site has not yet come back online, please give us a call at 256-547-6817 or email support@internetpro.net.   It is a holiday weekend, but if you hit 8 for support, you will get a technician.

 

 

 

2021-05-29T15:42:44-05:00May 29, 2021|News and Updates|Comments Off on Web Server Issue

Insecure PHP Notice

May 2021 – WordPress has a “Site Health” feature which makes recommendations about your host and site’s setup.   Recently, many users have been asking about a notice that they have seen in the dashboard starting that the server is running an insecure version of PHP and it should be upgraded.   This article addresses that notice.

The wording on those warnings is unfortunate. The servers displaying that are running Ubuntu 18.04 which is an LTS release (Long Term Support). The way Debian-based releases work (Ubuntu is Debian-based) is that they keep the versioning the same so as to guarantee compatibility throughout the life of the release, but they do patch the security issues. So, it is not actually an insecure release, but does behave has 7.2 PHP.  Most websites are currently on “shared servers” which are ones that house many sites. We run LTS releases because moving from PHP version to version frequently breaks *a lot*.  The newest sites being turned up have gone on 20.04 servers (and PHP 7.4 release), but there are currently issues with 7.4. For instance, email for forms, etc. does not support encryption like SSL or STARTTLS. We’re trying to get all of the bugs worked out of 7.4 before moving all of the servers up to the newest release and the usual schedule for doing that is in the 4th year of support – which in this case would be around April of 2022.

If you have any questions regarding this or if your site has a specific requirement for a PHP version greater than 7.2, please contact us at support@internetpro.net.

 

2021-05-19T06:59:37-05:00May 19, 2021|FAQ|Comments Off on Insecure PHP Notice

Trend Micro Hosted Email Security

May 10, 2021 – We have been receiving reports from customers using Trend Micro’s Hosted Email Security that are having delayed email.  The emails seem to be coming through, but they are very delayed. The issues started around 11:30am as that is the time of delivery for the last emails some customers are seeing.  Right now, we don’t have any news from Trend Micro regarding a time of resolution but we are working to find out.  Currently this issue is “up stream” of customer servers and is not a problem on their networks.

We will post updates here.

Update 2:24pm – They have posted this statement on their website:

–BEGIN
[ONGOING ISSUE] HES, TMES & PRE-FILTER | INBOUND MAIL LATENCY ISSUE
We had an issue in Hosted Email Security and Pre-filter earlier 2021-05-10 23:15 (UTC+8) where there’s a delay in delivery of emails affecting NABU, JP and EMEA region customer. We really apologize for the inconvenience it caused you but we are currently addressing the issue and will do our best to resolve it ASAP. Thank you for your understanding and patience on this matter.

–END

Update 4:26pm – We are starting to see pretty quick deliveries from Trend now.  According to them, no email was lost – just delayed.  It looks like this may now be resolved on their end, but it may take a bit for the email to be delivered.

 

2021-05-10T16:27:50-05:00May 10, 2021|News and Updates|Comments Off on Trend Micro Hosted Email Security
Go to Top