Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have nicer result failure messages #587

Closed
carl-baillargeon opened this issue Mar 18, 2024 · 5 comments · Fixed by #1027, #1029, #1031, #1032 or #1036
Closed

Have nicer result failure messages #587

carl-baillargeon opened this issue Mar 18, 2024 · 5 comments · Fixed by #1027, #1029, #1031, #1032 or #1036
Assignees

Comments

@carl-baillargeon
Copy link
Contributor

Would be nice to have a more human readable format for the test result failures messages:
image

BGP tests are also not very user friendly:
image

Could be related to #427

@carl-baillargeon
Copy link
Contributor Author

Adding VerifySnmpContact to the list:
image

@carl-baillargeon
Copy link
Contributor Author

@vitthalmagadum Will tackle the interfaces, LLDP, hardware tests.
@carl-baillargeon Will tackle the BGP tests.

@carl-baillargeon
Copy link
Contributor Author

Some tests failure messages will be improved as part of #388

@carl-baillargeon
Copy link
Contributor Author

@vitthalmagadum and @geetanjalimanegslab Please investigate remaining tests so we know how big of an effort it would be to remove the [ ] and { } in the failure messages.

Thanks

@geetanjalimanegslab
Copy link
Collaborator

geetanjalimanegslab commented Feb 4, 2025

Hi @carl-baillargeon
Here is the list of test files and testscases :

  1. test_generic.py - PR refactor(anta.tests): Nicer result failure messages generic routing module #1027
    1. VerifyRoutingTableSize
    2. VerifyRoutingTableEntry

  2. test_isis.py - fixed in PR fix(anta.tests): Cleaning up ISIS tests module #963

  3. . test_ospf.py PR - refactor(anta.tests): Nicer result failure messages OSPF test module  #1040
    1. VerifyOSPFNeighborState
    2. VerifyOSPFNeighborCount
    3. VerifyOSPFMaxLSA

  4. test_aaa.py PR refactor(anta.tests): Nicer result failure messages for AAA test module #1031
    1. VerifyTacacsServers
    2. VerifyTacacsServerGroups
    3. VerifyAuthenMethods
    4. VerifyAuthzMethods
    5. VerifyAcctDefaultMethods
    6. VerifyAcctConsoleMethods

  5. test_avt.py PR - refactor(anta.tests): Nicer result failure messages AVT, field_notice test module  #1036
    1. VerifyAVTSpecificPath

  6. test_bfd.py PR - refactor(anta.tests): Nicer result failure messages BFD & Connectivity test module #1029
    1. VerifyBFDPeersRegProtocols

  7. test_connectivity.py PR - refactor(anta.tests): Nicer result failure messages BFD & Connectivity test module #1029
    1. VerifyReachability
    2. VerifyLLDPNeighbors

  8. test_field_notices.py PR - refactor(anta.tests): Nicer result failure messages AVT, field_notice test module  #1036
    1. VerifyFieldNotice44Resolution

  9. test_hardware.py
    1. VerifyTransceiversManufacturers
    2. VerifyTransceiversTemperature
    3. VerifyEnvironmentPower

  10. test_interfaces.py
    1. VerifyInterfaceUtilization
    2. VerifyInterfaceErrors
    3. VerifyInterfaceDiscards
    4. VerifyInterfaceErrDisabled
    5. VerifyStormControlDrops
    6. VerifyPortChannels
    7. VerifyIllegalLACP
    8. VerifyLoopbackCount
    9. VerifyL3MTU
    10. VerifyL2MTU
    11. VerifyIPProxyARP
    12. VerifyInterfacesSpeed

  11. test_logging.py PR - refactor(anta.tests): Nicer result failure messages for Logging and security test module #1028
    1. VerifyLoggingHosts

  12. test_mlag.py PR - refactor(anta.tests): Nicer result failure messages MLAG test module  #1039
    1. VerifyMlagStatus
    2. VerifyMlagInterfaces
    3. VerifyMlagConfigSanity
    4. VerifyMlagReloadDelay
    5. VerifyMlagDualPrimary

  13. test_ptp.py PR- refactor(anta.tests): Nicer result failure messages PTP, software test module #1038
    1. VerifyPtpOffset
    2. VerifyPtpPortModeStatus

  14. test_security.py PR refactor(anta.tests): Nicer result failure messages for Logging and security test module #1028
    1. VerifySSHIPv4Acl
    2. VerifySSHIPv6Acl
    3. VerifyAPIHttpsSSL
    4. VerifyAPIIPv4Acl
    5. VerifyAPIIPv6Acl

  15. test_services.py PR - refactor(anta.tests): Nicer result failure messages Service and SNMP test module  #1041
    1. VerifyDNSServers

  16. test_snmp.py PR - refactor(anta.tests): Nicer result failure messages Service and SNMP test module  #1041
    1. VerifySnmpIPv4Acl
    2. VerifySnmpIPv6Acl
    3. VerifySnmpPDUCounters
    4. VerifySnmpErrorCounters

  17. test_software.py. PR: refactor(anta.tests): Nicer result failure messages PTP, software test module #1038
    1. VerifyEOSVersion
    2. VerifyTerminAttrVersion
    3. VerifyEOSExtensions

  18. test_stp.py PR: refactor(anta.tests): Nicer result failure messages STP and System test module  #1043
    1. VerifySTPMode
    2. VerifySTPBlockedPorts
    3. VerifySTPCounters
    4. VerifySTPForwardingPorts
    5. VerifySTPRootPriority
    6. VerifyStpTopologyChanges

  19. test_system.py PR: refactor(anta.tests): Nicer result failure messages STP and System test module  #1043
    1. VerifyCoredump
    2. VerifyNTPAssociations

  20. test_vxlan.py PR - refactor(anta.tests): Nicer result failure messages Vxlan test module #1032
    1. VerifyVxlanConfigSanity
    2. VerifyVxlanVniBinding
    3. VerifyVxlanVtep

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment