Siemens CP343-1 Lean RX/TX stop working

[nico14]

Hi everyone,

I have recently installed Siemens CP343-1 lean Ethernet modules (firm V3) on existing production line machine PLCs. Installation of the modules went fine, and I am able to now see the entire plant using Siemens PRONETA.

The purpose of the install was to:
1) Upgrade the network from MPI to Ethernet
2) Connect all production line machines to a Kepware OPC server which in turn talks to an OPC UA client that is part of a new OEE/quality production monitoring system.

Following the exact same install process from a S7 hardware perspective, along with identical Channel/PLC setup in Kepware, I am now able to extract data via direct addressing from machine PLCs via the Kepware server (no connections from the machine PLC CPU to the OPC server were needed to be created in Netpro.)

My problem is the following:
Out of the 24 PLCs that are connected to the OPC server, 4 PLCs connections to the server constantly drop out.

Upon inspection of the CP343-1 lean units, the 5V and Run LED are lit solid Green, along with Port X1 LED lit solid orange. The RX/TX light is not flashing, and Ethernet coms to the PLC cease. Using MPI, I have checked both the CPU and CP card online for any diagnostics. No error exist.

Cycling power on the PLC re-establishes RX/TX functionality to the CP343-1 lean module, and I am able to see the unit once again in the OPC server. However after a non constant time period, RX/TX function stops and I lose coms to the server.

I am at a loss to what is causing the issues.

Further info:
Each line is connected as a STAR system with all Ethernet coms being sent through a single X200 series switch. This central line control switch is connected to the server via fibre through a media convertor.

All parts are Siemens.

Any help would be greatly appreciated.
Cheers

 

[Bratt]

My first guess is cable problems. Have you tried to switch cable on one of the faulty connections?

How long are the cables from the switch to the CP card? 100 meters is maximum recommended length

Have you checked the log in the switch? Since it is manged you will have a built in webpage. Some switches also have more extensive port monitoring that you can activate perhaps that reveals something.

 

[JesperMP]

Can you ping the affected CPs ? Both locally directly connected, and remotely from the OPC server PC ?
edit: When checking ping locally, remove the cable to the OPC server.

Can you go online via the affected CPs ?

Check the affected PLCs online via MPI, investigate the connection resources. If all connections are used, then maybe connections are not released properly. You could try to use an alternative OPC Server instead of Kepware. If you use Simatic Net as OPC Server it may be an advantage to create a fully specified connection rather than an unspecified connections as with Kepware (I assume this).

edit: Also try to swap an affected CP with one of the unaffected CPs. Then do the error follow the CP or does it follow the PLC ?

 

[Bratt]

Is cycling the power the only way to reestablish communication? Have you tried unplugging and replugging the cable?

When the communication is down have you tried to ping the faulty cp card?

 

[Bratt]

Check the affected PLCs online via MPI, investigate the connection resources. If all connections are used, then maybe connections are not released properly.

Perhaps but even if all resources are exhausted it should still respond to ping and the RX/TX light should not go out completely.

 

[nico14]

Thankyou all for your replies.

Bratt:
1) All cables are tested and are ok. I ran duel cables to each CP as fail safe just in case a cable was damaged during install.
2) All cables are within max cable lengths as per Cat5e Siemens profinet standards. The units in question are also the closest to the line switch locations (max 50m).
3) There are no logged errors in the X200 series switches. All ports are RX/TX enabled, along with full duplex 100mbps and Autonegotiation.
4) Cycling power on the CPU is the only way to re-establish RX/TX function on the CP. Once this is done, the RX/TX LED starts to flash again.


JesperMP:
1) When the CP is functioning as per normal, ie: RX/TX light is flashing, I am able to PING the CP from my OPC server. When the error occurs however I am not able to PING the CP UNIT form the server, nor am I able to PING the CP UNIT from a PG.

2) Using MPI to check both CPU and CP diagnostics I find the following;
CPU
a) No faults listed in the CPU diagnostics
b) Adequate connection resource have been allocated for PG/OP/S7 and other communications.
c) Cycle times of the CPU are around 35ms whilst I am only retrieving data every 2000ms via the OPC server.

CP unit
a) Whilst RX/TX light is not flashing (ie not working), unable to bring up full compliment of unit info. An error message appears when trying to bring up the module information from the S7 manager. What loads is a single information tab with no errors listed. A running status is also not available. Trying to load special diagnostics fails with a coms error.
b) After cycling power on the PLC rack, RX/TX starts again and I am able to monitor the full compliment of features of the CP card in module information and special diagnostics. However, no error or fault is listed to give any indication on what is causing CP unit RX/TX to stop.

3) Multiple CPs have been tried but the issue remains at the same machine location. However I do not believe it is CPU related as the machines dropping out of the network have varied. A few machines are constant, but others have varied.

4) I initially started with Simatic net but moved to Kepware as Simatic net OPC UA would not work with the UA client of the production monitoring system. Kepware worked first go and was far easier to use.
I don't believe a specified or unspecified connection will make a difference as I would be experiencing the same issue across the plant. Also Kepware does not require connection to be established in netpro to the server.

I've tried a lot of things but without any luck.

Cheers

 

[Bratt]

Seems like the CP malfunctions but usually when you get a serious error on the CP all leds starts to flash.

Can it be an adress conflict? Is the IP set by DHCP or fixed. Can you test an ip adress conflict set your laptop to the same IP as the cp and see what happens.

Where is the CP located in the RACK? You can test and change address of the CP to make sure that the CPU is not writing to that address by mistake

Do you have another cp type a regular 343 that you can test with?

 

[Bratt]

Do the faults happen at random times or the same time?

Some other long shots you can try.

Auto negotiation have caused me problems with some brands you can try to disable auto negotiation set it to fixed 100/100 or even try half duplex or go as low as 10/10.

Disable SNMP in the CP card.

Disable the web server in the CP card.

Disable time synchronization.

Change ip adress on the cp card.

Upgrade to Firmware 3.1
Downgrade to Firmware 2.6

Change Input and output address for the CP card.

Try to place the CP in the slot directly after the CPU.

Also check what Jesper said and connect to the CPUS while they are working and check module diagnostics number of active connections and make sure that that number is constant and not increasing

If the problem is caused by malformed packet or something else on the network your best chance is to activate port mirroring and mirror all traffic to/from on of the CPs that you know faults out and connect a computer with Wireshark and start recording until fault appears.

 

[nico14]

Thanks Bratt,

I did some work on the network yesterday and in the end, the most successful action I took was to remove a legacy data harvesting PLC from the network. Once removed, all RX/TX lights on all CP cards began to flash at a frequency that was constant with the frequency they were being polled at.

The network became more stable and I have not experienced loss in coms to any CP cards bar one. What is note worthy is that the one CP unit I lost connection to has a specified connection to another PLC on the network who's CP343-1 lean card is constantly transmitting data (ie: its RX/TX light is lit constantly).

Investigating this unit further I found the following:
a) Online web based diagnostics -> TCP/IP statistics -> received data packets rejected to checksum or alignment errors found on both ports 1 and 2.
b) CPU firmware V3.27, CP343-1 lean firmware V2.6, CP343-1 firmware
V3.0.23
c) CPU has an I/O access writing error
d) 1 extra S7 and other connection is being utilised that has not been reserved on the CPU.

My gut feeling at this stage is that a device on the network is over polling and "crashing" coms to certain PLC's. I believe the CPU's on the affected units may be locking down its respective CP cards to protect the CPU from over polling or excess coms traffic???

To diagnose this ill do as you recommend and use some software to monitor network traffic.

Once I have some more information I will be sure to post it.

Cheers

 

[JesperMP]

The CP343-1 has to pass all its comms via the slow MPI backplane of the S7-300 rack - 187.5 kbps shared with all other devices on the MPI network.
So it is easy to overload the CP343-1 with too much comms.
This is something to consider with all CPs in S7-300 systems.

 

[med ali haloui]

connection opc and s7300 cpu3152DP

I have a problem with the communication virtual between S7300 CPU3152DP and opc
How is port communication done?
I want a virtual communication and I have not found any information