CNIs and Cisco 3850

One of the sites we work at has recently undergone a major upgrade of the site network, which included upgrading the switches to Cisco 3850.

Coinciding with those works we started to see issues talking to our CNIs, which were mostly Mk1 but includes some Mk2. Typically we'd start getting packet loss which pinging them, and that would increase over time from a few percent to much higher (up to 100%). C-Bus comms would become erratic or stop.

If they plug their laptops into the switches there's no packet loss, so predictably "it's our kit that's faulty", despite it having worked flawlessly for years prior to the upgrade, and us being able to tell them from our own logs when they changed the hardware over based entirely on when we started getting comms issues to the CNIs. Indeed we've even proven in one case that putting a cheap desktop switch between the CNI and the Cisco fixes the issue, but they won't entertain doing any more of that for the valid reason that it's not the right solution, but still falling back to the (I believe wrong) conclusion that it's up to us to fix the issue because the CNIs are the problem.

Any suggestions?

We're going to site next week to swap out a couple of CNIs and test the "faulty" ones on a desk so any suggestions of tests we should add into the mix would be welcomed. Unfortunately I think we'll have limited support from IT on site because they've all but washed their hands of it.

 

Hi Mark,

Look at the port speed negotiation of the Cisco 3850 port and have them set it to noneg and hard set the Ethernet speed.

I think the CNI1 will require Half Duplex 10MBps and the CNI2 can be Full Duplex 100Mbps.

Here is the config guide for their IT guys.

Hope that helps,

Brad

 

Hey Mark,

I agree with Brad, turning off the auto neg does help, the Ciscos are for some reason known for doing it badly.

You may have to unfortunately replace your older CNI1s with the 2s though. From memory we had quite a hard time getting the Ciscos to play with the 1s. This was even with great help from the IT department. The 2s just worked (yes we still had to turn off the auto negotiation)

We have a site where we have about 5 CNIs running on a full corporate network with pure Cisco and did also have some teething issues, however it has now been running for quite some time now with no problems.

J

 

Thanks for this suggestion. I have a CNI 1 in my office I can try this on.

Looking at the Lantronix website I see several possible candidates for CoBox:
https://www.lantronix.com/resources/discontinued-products/

Which files should I be looking at?

Many of them are this old if not considerably older in some cases so it's certainly worth a look.

Updated to add: My office test CNI's web interface just says Lantronix Web-Manager 3.30. having spent ages getting the java app to actually run all I now get is a blank screen. It's been so long since I used that web interface I can't even remember what I'm supposed to see there!

 

Thanks (and also to Brad), I'll get site to try this although I know we did try it before on one site without much success - that was just one of many things we tried in a fairly unscientific scattergun approach though so I'm not confident that we proved anything.

Re: the "Ciscos are for some reason known for doing it badly" - do you have any reference to this? It would help quite a lot to try to meet in the middle on the hardware blame game; replacing CNIs and having settings changed to make things work all plays into "the CNIs are at fault" argument.

We certainly have several working, but they seem to be deteriorating over time. (The switch upgrade took place in December/January.) Is there any reason why they might sometimes work then stop working?

 

This thread may be of help with the firmware upgrading.

http://www.cbusforums.com/forums/showthread.php?t=11869&highlight=Cobox

Rohan

 

Hey, Not particually, its just a thing that you hear often when you hang around the networking guys for too long.

At the end of the day, it is probably the NIC on the Lantronics card not quite doing things right for this day and age. Hopefully the firmware update does it for you. It worked for us to get the funding to replace the CNIs and look when they are 12+ years anyway they were due for replacement from a maintenance point of view.

In terms of the decay, dunno, but I have seen that happen before. After a power cycle they seem ok again for a while but then decay, probably the caps on the NIC transformers are dying.

This link is worth looking at if you are interested:
http://www.cisco.com/c/en/us/suppor...-series-switches/17053-46.html#auto_neg_valid

 

Thanks for that, I've successfully upgraded one in my office from v5.2 to v5.8, and DeviceInstaller looks like a useful tool to have around, at least while I have any old CNIs.

That process lead me to dsearch.exe. Running that on site listed all my old CNIs, all of CNIs have the same old v5.2 firmware (as expected) except one, which has v5.9 (later than any I can see on the Lantronix website). Anyone here know anything about v5.9?

Playing with Cisco port settings didn't get us anywhere and the worst offending CNIs have now been replaced with v2, so I think the plan now is to see if we have problems in future and upgrade the firmware as a first step if we do. (I don't want to risk making things worse by upgrading firmware on CNIs which are currently working.)

Thanks all for the help and interesting info; I'll update here if I discover anything that would be of use to others.

Mark

 

The Lantronix website suggests that the latest for the Cobox-Mini100 is 5.8.0.1 rather than 5.8.0.5, although as you mention the link is broken. However it is on their FTP site:
ftp://ftp.lantronix.com/pub/cobox-mini100/5.8.0.1/

 

Hey Mark,

Just wondering how you went with this? Would be interested if you managed to get anything that actually pointed to a cause.

J

 

This is still very much a live issue, and we're getting Cisco involved, however further investigation has shown some very similar issues on the old switches as well as the new ones, so I think it's either configuration or (switch) software update related. It ran for >10 years on the old switches just fine, but we have 2 Mk1 CNIs and 1 Mk2 CNI right now that we can't talk to...

 

This sounds like a serious problem for you.
I wonder if an option would be to consider a serial device server connected to CBus RS232 interface.
It's my preferred way to connect to a CBus network and I think these devices are great.
https://www.moxa.com/product/NPort_5100A.htm
I think it would be at least worth a try.

 

Hmm very interesting!

As I have mentioned we have this setup at a number of big sites and certainly have not had the severity of the issues you are having, which as you say makes me think it is a config or network infrastructure issue.

I would normally think that there was something wrong in the IGMP sniffing but the lantronics boxes are setup for TCP server/client so probably not.

 

People can say all they like how Lantronix interface is OK but my experience is going back at least a decade and I think they are better off avoided.
Setup, which might be a lot better now caused all sorts of grief and you can search the forums from 10 years ago to see that.
Regardless Clipsal CNI is a pretty robust device but the first thing that is going to give trouble is the ethernet interface.

 

I've not had many issues with the Lantronix ones over the years, it's only really since Jan after switch upgrades that we've been stung. But it's odd that we've had some similar issues (although much less often) with MK2 which aren't Lantronix[*]. And always, putting a cheap desktop switch between the CNI and the Cisco "fixes" things.

We're very much in the hands of site's IT on this but they have sent packet traces to Cisco so hopefully someone will come up with something.

It's (supposed to be) a fully redundant system with two sets of CNIs, and we've been able to shuffle things around so that we only have failures in places where we also have a working one, so the system as a whole is working (albeit without the redundancy in places). Sadly that means that the eagerness to get involved and sort things out is somewhat reduced.

[*] It is quite possible that we have different issues with similar symptoms, of-course.