Telco – Clear to the smart jack!

While recently troubleshooting a T1 circuit issue for a MPLS connection I was getting the run around from my service provider. The classic phrase we have all heard. “Cleared on Test”, “Clear to the smart jack”, and “Testing clean on loop” as well as other variations are common phrases I hear. Telco is all to quick to dismiss my complaints and they always think they have no issues.

What if I am testing clean on loop? What if I am clearing on test? Who do we point the finger to then?

Well that is exactly what I had happen to me. Telco never saw the input and crc errors that I was getting on my serial interface. They could loop smart jack telco facing and test clean for all patterns. 0x1111 was giving me the most trouble. I could put my gear in hardware loop and send 0x1111 all day long just fine. But when I was connected to telco I would get about a 28% success rate. I knew it would be a cabling or telco issue. So I changed the patch cable between smart jack and CSU. After the 3rd patch cable the issue remained.

Telco stuck to their guns telling me to change the cable. Finnaly after getting telco and the various local exchange carriers on the line at the same time, with the backing of my hardware vendor I was able to find the issue. I had them loop customer facing from “test point” which happened to be on the telco side of the smart jack, about a 60 miles from my gear. Somewhere in this 60 miles was an issue. I was determined to find it.

I asked telco to loop the smart jack customer facing. They couldn’t do this. How convenient that they can loop telco facing to prove it isn’t their issue but cannot loop customer facing to prove the customer doesn’t have a problem. So I did the next best thing. I had them but a coupler on the end of my patch cable before it plugs into the smart jack. I then was able to test clean on all test patterns, 0x1111 included. This ruled out my patch cable and my gear.

The LEC who was onsite assisting this troubleshooting happened to have a smart jack. I asked that he install it so we could rule out the smart jack. I had the telco in the major city close to this office give me a loop customer facing from the test point. Sent my test patterns and tested clean!!! It was a telco issue, Imagine that. A bad smart jack. The kicker was the smart jack had been changed 4 days before hand in attempt to resolve the errors. Bad batch of smart jacks possibly.

So the next time the telco has you running around to prove it isn’t you. Stick to YOUR guns. Clear to smart jack doesn’t always rule them out!

3 Responses

  1. Derek Eugea Says:

    Great post. I came across this post after doing a search on google for signs of a bad smart jack.

    I am currently in this same situation. A new remote site’s MPLS circuit went down after a week and a half of use with no problems. Yesterday, all of a sudden the connection goes down. I call my service provider (Mitel Netsolutions) and they troubleshoot the line with the LEC (AT&T btw). So the LEC does their testing and the line comes back up for about an hour and then goes back down (I assume it went back down because of additional testing). I then get a call from the MPLS Provider and they say the LEC says it is clear to the Smart-Jack and it is the CSU/DSU. So that was yesterday evening. So some time during the night the circuit comes back up and the remote site is working fine until about an hour ago when the line went down again. I call the provider they say well the LEC says it is your equipment we need to reboot your router and reset your CSU/DSU WIC first. So I do all this, and now I have no Carrier Detect light on my CSU/DSU. I have been doing this for 7+ years and have never had an issue with a CSU/DSU being bad and was wondering what your thoughts were? Thanks for any incite on the subject.

  2. admin Says:

    I would have them replace the smart jack.

  3. IAMTELCO Says:

    So I am the dreaded TELCO.

    Yep…he’s right. I work in the NOC of a Billion dollar not AT&T but another big one.

    And I have on multiple occasions told our customers – Clean to the Smart Jack and I can say no more than 10 instances over the past 3 years has the issue actually been the SmartJack.
    So yes this does occur, though infrequent enough to not be the first thing we go to since there are plenty more common reasons for the above symptoms.

    Understand that when 90% of cases where Clean to the Smartjack really means Clean to the Smartjack, it is very easy to ASSUME that the issue is going to be on the customer’s gear or cabling.
    Also looping the CSU is a feature that not all TELCO’s have. When they don’t have that ability, only a local dispatch to do what is called a HARD LOOP and head to head testing can be done to prove out the smart jack.

    AT&T and most LEC have the ability to loop the NIU(Smartjack) and the CSU(customer facing side of of NIU).

    CLEC on the other hand(TW, Comcast, Grande, TelePacific) usually only have the ability to loop the NIU or their gear the Demarcation cable is plugged into(usually a router of some kind)

    So when you come across this issue (Especially if you use a PRI) ask your Telco Provider to dispatch locally and prove out services to the SmartJack.

    Beware – this is usually billable and if tests run clean prepare to pay a minimum of about $150 for the visit.

    I hope this helps.


Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.