Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issues for changing node_id for stratum, how to fix it? #987

Open
xfyan0408 opened this issue Jun 25, 2022 · 3 comments
Open

Issues for changing node_id for stratum, how to fix it? #987

xfyan0408 opened this issue Jun 25, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@xfyan0408
Copy link

xfyan0408 commented Jun 25, 2022 •
edited
Loading

On the ONOS end, I need to identify different deviceID of stratum-based switch. I only changed the nodeID in the chassis_config_pb.txt file to 96.

chassis {
  platform: PLT_GENERIC_BAREFOOT_TOFINO
  name: "Edgecore Wedge100BF-32x"
}
nodes {
  id: 96
  slot: 1
  index: ### 1
}
singleton_ports {
  id: 1
  name: "1/0"
  slot: 1
  port: 1
  speed_bps: 10000000000
  config_params {
    admin_state: ADMIN_STATE_ENABLED
  }
  node: 96
}

When starting stratum, I found error logs, how should I solve this problem,
image

This seems that nodeID cannot be changed, but I did it successfully on earlier versions of stratum (corresponding to stratum of BF-SDE-9.2.0).

Does someone know this? too much thx!
@xfyan0408 xfyan0408 added the bug Something isn't working label Jun 25, 2022
@xfyan0408
Copy link
Author

xfyan0408 commented Jun 25, 2022 •
edited
Loading

It seems that it can not change node id for stratum, how to fix it?

@Dscano
Copy link
Contributor

Dscano commented Jun 25, 2022

Hi linhil,

The nodeID in the switch should be 0 or 1, I think that is a design choiche is defined in the P4 standard. You can change the deviceID in ONOS changing the string in the netcfg file, more precisely here https://github.com/stratum/fabric-tna/blob/fd3c3f0989ae6c2bb2384bdd1b5b7755168ba886/tofino-netcfg.json#L3

@daniele-moro daniele-moro mentioned this issue Jun 27, 2022
Closed
@pudelkoM
Copy link
Member

I confirmed that the node ID can be picked at users discretion.

chassis_config.pb.txt

description: "Default Chassis Config for Edgecore Wedge100BF-32x"
chassis {
  platform: PLT_GENERIC_BAREFOOT_TOFINO
  name: "Edgecore Wedge100BF-32x"
}
nodes {
  id: 2
  slot: 1
  index: 1
}
singleton_ports {
  id: 1
  name: "1/3"
  slot: 1
  port: 1
  channel: 4
  speed_bps: 10000000000
  config_params {
    admin_state: ADMIN_STATE_ENABLED
    autoneg: TRI_STATE_FALSE
  }
  node: 2
}

Then connect with device ID 2:
./p4runtime-sh-docker --config=<...> --grpc-addr=10.70.10.94:28000 --device-id=2

Logs:

bf_kdrv_mod found! Unloading first...
loading bf_kdrv_mod...
I20220630 21:45:10.167529 10084 logging.cc:72] Stratum version: not stamped.
I20220630 21:45:10.168002 10084 bf_sde_wrapper.cc:1757] bf_sysfs_fname: /sys/class/bf/bf0/device/dev_add
Install dir: /usr (0x3bd8020)
bf_switchd: system services initialized
bf_switchd: loading conf_file /usr/share/stratum/tofino_skip_p4.conf...
bf_switchd: processing device configuration...
Configuration for dev_id 0
  Family        : Tofino
  pci_sysfs_str : /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.0
  pci_domain    : 0
  pci_bus       : 5
  pci_fn        : 0
  pci_dev       : 0
  pci_int_mode  : 1
  sbus_master_fw: /usr/
  pcie_fw       : /usr/
  serdes_fw     : /usr/
  sds_fw_path   : /usr/share/tofino_sds_fw/avago/firmware
  microp_fw_path:
bf_switchd: processing P4 configuration...
P4 profile for dev_id 0
  p4_name: dummy
    libpd:
    libpdthrift:
    context:
    config:
  Agent[0]: /usr/lib/libpltfm_mgr.so
  diag:
  accton diag:
  non_default_port_ppgs: 0
  SAI default initialize: 1
bf_switchd: library /usr/lib/libpltfm_mgr.so loaded
bf_switchd: agent[0] initialized
Health monitor started
Operational mode set to ASIC
Initialized the device types using platforms infra API
ASIC detected at PCI /sys/class/bf/bf0/device
ASIC pci device id is 16
bf_switchd: drivers initialized
Skipping P4 program load for dev_id 0
Setting core_pll_ctrl0=cd44cbfe
2022-06-30 21:45:18.348978 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349005 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349022 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349035 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349062 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349076 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349089 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:18.349102 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage

bf_switchd: dev_id 0 initialized

bf_switchd: initialized 1 devices
Skip p4 lib init
Skip mav diag lib init
bf_switchd: spawning cli server thread
bf_switchd: spawning driver shell
bf_switchd: server started - listening on port 9999
I20220630 21:45:20.305521 10084 bf_sde_wrapper.cc:1767] switchd started successfully
I20220630 21:45:20.305662 10084 main_bfrt.cc:50] Detected is_sw_model: 0
I20220630 21:45:20.305699 10084 main_bfrt.cc:51] SDE version: 9.7.2
Cannot read termcap database;
using dumb terminal settings.
I20220630 21:45:20.305723 10084 main_bfrt.cc:52] Switch SKU: TOFINO_32D, revision B0, chip_id 0x83058b061255c85
W20220630 21:45:20.306016 10084 credentials_manager.cc:59] No key files provided, using insecure server credentials!
bf-sde> W20220630 21:45:20.307399 10084 credentials_manager.cc:78] No key files provided, using insecure client credentials!
I20220630 21:45:20.308743 10084 timer_daemon.cc:91] The timer daemon has been started.
I20220630 21:45:20.308969 10084 hal.cc:127] Setting up HAL in COLDBOOT mode...
I20220630 21:45:20.309191 10084 config_monitoring_service.cc:94] Pushing the saved chassis config read from ./chassis_config_alt_node_id.pb.txt...
I20220630 21:45:20.324800 10084 bfrt_switch.cc:321] Chassis config verified successfully.
I20220630 21:45:20.326619 10084 phal.cc:94] No phal_config_file specified and no switch configurator found! PHAL will start without any data source backend. You can specify "--define phal_with_tai=true" while building to enable TAI support, or "-enable_onlp" at runtime to enable the ONLP plugin.
I20220630 21:45:20.327490 10084 attribute_database.cc:207] PhalDB service is listening to localhost:28003...
I20220630 21:45:20.327567 10084 bf_chassis_manager.cc:1404] Successfully registered port status notification callback.
I20220630 21:45:20.338596 10084 bf_chassis_manager.cc:111] Added port 1 in node 2 (SDK Port 131).
I20220630 21:45:20.338644 10084 bf_chassis_manager.cc:128] Set autoneg policy TRI_STATE_FALSE for port 1 in node 2 (SDK Port 131).
I20220630 21:45:20.342468 10084 bf_chassis_manager.cc:147] Enabled port 1 in node 2 (SDK Port 131).
I20220630 21:45:20.342634 10084 bfrt_switch.cc:60] Chassis config pushed successfully.
I20220630 21:45:20.342674 10084 gnmi_events.h:709] Handling stratum::hal::ConfigHasBeenPushedEvent
I20220630 21:45:20.342705 10084 gnmi_publisher.h:205] Configuration has changed.
I20220630 21:45:20.345501 10084 p4_service.cc:121] Pushing the saved forwarding pipeline configs read from /etc/stratum/pipeline_cfg.pb.txt...
I20220630 21:45:21.068913 10084 bfrt_switch.cc:277] P4-based forwarding pipeline config verified successfully for node with ID 2.
bf_switchd: starting warm init for dev_id 0 mode 1 serdes_upgrade 0
bf_switchd: agent[0] library unloaded for dev_id 0
bf_switchd: library /usr/lib/libpltfm_mgr.so loaded
bf_switchd: agent[0] initialized
Health monitor started
Skip diag lib deinit
\2022-06-30 21:45:22.098339 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098365 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098381 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098394 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098407 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098419 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098433 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:22.098445 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
I20220630 21:45:24.803139 10084 bf_sde_wrapper.cc:1929] Registered packetio callbacks on device 0.
I20220630 21:45:24.814724 10084 bf_chassis_manager.cc:1089] Set MTU 0 for port 1 in node 2 (SDK Port 131).
I20220630 21:45:24.814771 10084 bf_chassis_manager.cc:1096] Set autoneg policy TRI_STATE_FALSE for port 1 in node 2 (SDK Port 131).
I20220630 21:45:24.814793 10084 bf_chassis_manager.cc:1104] Set loopback mode LOOPBACK_STATE_UNKNOWN for port 1 in node 2 (SDK Port 131).
I20220630 21:45:24.818393 10084 bf_chassis_manager.cc:1113] Enabled port 1 in node 2 (SDK Port 131).
I20220630 21:45:24.818416 10084 bf_chassis_manager.cc:1123] Replayed port 1 in node 2.
I20220630 21:45:24.818538 10084 bf_chassis_manager.cc:1171] Replayed chassis config for node 2.
I20220630 21:45:24.818554 10084 bfrt_switch.cc:78] P4-based forwarding pipeline config pushed successfully to node with ID 2.
E20220630 21:45:24.821306 10084 hal.cc:220] Stratum external facing services are listening to 0.0.0.0:28000, localhost:9559...
I20220630 21:45:26.415447 10325 bf_chassis_manager.cc:1288] State of port 1 in node 2 (SDK port 131): UP.
I20220630 21:45:26.415500 10322 gnmi_events.h:709] Handling stratum::hal::PortOperStateChangedEvent
I20220630 21:45:36.358637 10414 sdn_controller_manager.cc:148] New SDN connection (role_name: <default>, election_id: { high: 1 }, uri: ipv4:10.70.10.200:48878): device_id: 2 election_id { high: 1 }
I20220630 21:45:36.356567 10414 sdn_controller_manager.cc:323] Informing all connections about primary connection change.
I20220630 21:45:36.356878 10414 sdn_controller_manager.cc:199] New primary connection for role <default> with election ID { high: 1 }.
I20220630 21:45:36.356958 10414 p4_service.cc:590] Controller (role_name: <default>, election_id: { high: 1 }, uri: ipv4:10.70.10.200:48878) is connected as MASTER for node (aka device) with ID 2.
I20220630 21:45:36.393503 10567 bfrt_switch.cc:277] P4-based forwarding pipeline config verified successfully for node with ID 2.
bf_switchd: starting warm init for dev_id 0 mode 1 serdes_upgrade 0
bf_switchd: agent[0] library unloaded for dev_id 0
bf_switchd: library /usr/lib/libpltfm_mgr.so loaded
bf_switchd: agent[0] initialized
Health monitor started
/2022-06-30 21:45:37.405411 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405432 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405456 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405468 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405479 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405490 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405502 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
2022-06-30 21:45:37.405513 BF_TM  WARN - Traffic Manager buffer pool 0 pfc limit is higher or equal to pool size. PFC might not trigger based on pool usage
I20220630 21:45:40.116822 10567 bf_sde_wrapper.cc:1929] Registered packetio callbacks on device 0.
I20220630 21:45:40.127763 10567 bf_chassis_manager.cc:1089] Set MTU 0 for port 1 in node 2 (SDK Port 131).
I20220630 21:45:40.127830 10567 bf_chassis_manager.cc:1096] Set autoneg policy TRI_STATE_FALSE for port 1 in node 2 (SDK Port 131).
I20220630 21:45:40.127851 10567 bf_chassis_manager.cc:1104] Set loopback mode LOOPBACK_STATE_UNKNOWN for port 1 in node 2 (SDK Port 131).
I20220630 21:45:40.131418 10567 bf_chassis_manager.cc:1113] Enabled port 1 in node 2 (SDK Port 131).
I20220630 21:45:40.131440 10567 bf_chassis_manager.cc:1123] Replayed port 1 in node 2.
I20220630 21:45:40.131565 10567 bf_chassis_manager.cc:1171] Replayed chassis config for node 2.
I20220630 21:45:40.131584 10567 bfrt_switch.cc:78] P4-based forwarding pipeline config pushed successfully to node with ID 2.
I20220630 21:45:41.727454 10325 bf_chassis_manager.cc:1288] State of port 1 in node 2 (SDK port 131): UP.
I20220630 21:45:41.727514 10322 gnmi_events.h:709] Handling stratum::hal::PortOperStateChangedEvent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pudelkoM @Dscano @xfyan0408