xsdata creates two classes for the same type #1106

amal-khailtash · 2024-12-08T01:45:00Z

I am not sure what is going on, but there seems to be two dataclasses defined from the same type. The same problem shows up in different schemas from here: http://www.accellera.org/XMLSchema

Specifically the problematic type is portAccessType defined in:

http://www.accellera.org/XMLSchema/SPIRIT/1.5/port.xsd
http://www.accellera.org/XMLSchema/SPIRIT/1685-2009/port.xsd
http://www.accellera.org/XMLSchema/IPXACT/1685-2014/port.xsd
http://www.accellera.org/XMLSchema/IPXACT/1685-2022/port.xsd

They all show the same issue.

I see two python files (port_access_type.py and port_access_type_1.py) generated with classes PortAccessType and PortAccessType1 respectively. Both of these have:

class Meta:
    name = "portAccessType"

Here are the two files:
port_access_type.py

from dataclasses import dataclass, field
from typing import Optional

from org.accellera.ipxact.v1685_2014.simple_port_access_type import (
    SimplePortAccessType,
)

__NAMESPACE__ = "http://www.accellera.org/XMLSchema/IPXACT/1685-2014"


@dataclass(slots=True)
class PortAccessType:
    """Indicates how a netlister accesses a port.

    'ref' means accessed by reference (default) and 'ptr' means accessed
    by pointer.
    """

    class Meta:
        name = "portAccessType"
        namespace = "http://www.accellera.org/XMLSchema/IPXACT/1685-2014"

    value: Optional[SimplePortAccessType] = field(
        default=None,
        metadata={
            "required": True,
        },
    )

port_access_type1.py

from collections.abc import Iterable
from dataclasses import dataclass, field
from typing import Optional

from org.accellera.ipxact.v1685_2014.leaf_access_handle import LeafAccessHandle
from org.accellera.ipxact.v1685_2014.port_access_type import PortAccessType

__NAMESPACE__ = "http://www.accellera.org/XMLSchema/IPXACT/1685-2014"


@dataclass(slots=True)
class PortAccessType1:
    """
    :ivar port_access_type: Indicates how a netlister accesses a port.
        'ref' means accessed by reference (default) and 'ptr' means
        accessed through a pointer.
    :ivar access_handles:
    """

    class Meta:
        name = "portAccessType"

    port_access_type: Optional[PortAccessType] = field(
        default=None,
        metadata={
            "name": "portAccessType",
            "type": "Element",
            "namespace": "http://www.accellera.org/XMLSchema/IPXACT/1685-2014",
        },
    )
    access_handles: Optional["PortAccessType1.AccessHandles"] = field(
        default=None,
        metadata={
            "name": "accessHandles",
            "type": "Element",
            "namespace": "http://www.accellera.org/XMLSchema/IPXACT/1685-2014",
        },
    )

    @dataclass(slots=True)
    class AccessHandles:
        access_handle: Iterable[LeafAccessHandle] = field(
            default_factory=list,
            metadata={
                "name": "accessHandle",
                "type": "Element",
                "namespace": "http://www.accellera.org/XMLSchema/IPXACT/1685-2014",
                "min_occurs": 1,
            },
        )

I see a number of duplicate files with *1.py, *2.py with the same class Meta names!

http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/

http://www.accellera.org/XMLSchema

Here are the commands I use to generate the binding classes.

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1_5 http://www.accellera.org/XMLSchema/SPIRIT/1.5/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009 http://www.accellera.org/XMLSchema/SPIRIT/1685-2009/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2014 http://www.accellera.org/XMLSchema/SPIRIT/1685-2014/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2022 http://www.accellera.org/XMLSchema/SPIRIT/1685-2022/index.xsd



$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009.ve http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009.ve.ams http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/AMS/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009.ve.core http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/CORE/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009.ve.pdp http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/PDP/index.xsd

$ xsdata generate --slots --subscriptable-types --generic-collections --relative-imports --structure-style filenames --package org.accellera.spirit.v1685_2009.ve.power http://www.accellera.org/XMLSchema/SPIRIT/1685-2009-VE-1.0/POWER/index.xsd

I am not sure if the problem is with the XSDs or a bug in xsdata.

The text was updated successfully, but these errors were encountered:

tefra · 2024-12-08T05:43:40Z

It's quite common vendors to give the same name to complex types and elements, xsdata has to add these suffixes to avoid duplicate class names.

<xs:complexType name="portAccessType">
  <xs:sequence>
    <xs:element ref="spirit:portAccessType" minOccurs="0">
    <xs:annotation>
      <xs:documentation>Indicates how a netlister accesses a port. 'ref' means accessed by reference (default) and 'ptr' means accessed through a pointer.</xs:documentation>
    </xs:annotation>
    </xs:element>
    <xs:element ref="spirit:portAccessHandle" minOccurs="0"/>
  </xs:sequence>
</xs:complexType>

amal-khailtash · 2024-12-08T15:22:53Z

That is fair, but xml serialization is failing on valid xml on these duplicates. Somehow they are mixed up.

amal-khailtash · 2024-12-08T17:01:38Z

Here are some problematic xml files that fail to deserialize:

xml.tar.gz

For scmladaptor.xml, I get:

.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:67: in from_string
    return self.from_bytes(source.encode(), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:88: in from_bytes
    return self.parse(io.BytesIO(source), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:59: in parse
    result = handler.parse(source, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:41: in parse
    return self.process_context(ctx, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:59: in process_context
    self.parser.start(
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:95: in start
    child = item.child(qname, attrs, ns_map, len(objects))

>           raise ParserError(f"Unknown property {self.meta.qname}:{qname}")
E           xsdata.exceptions.ParserError: Unknown property {http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType:{http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType

.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/nodes/element.py:469: ParserError


E           Failed: Failed to deserialize XML file 'tests/xml/ipxact/v1685_2022/Leon2/NXP.com/adaptor_scml/scmladaptor.xml' using class '<class 'org.accellera.ipxact.v1685_2022.abstractor.Abstractor'>': Unknown property {http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType:{http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType

Similar issue with component types. For uart_scml.xml:

.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:67: in from_string
    return self.from_bytes(source.encode(), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:88: in from_bytes
    return self.parse(io.BytesIO(source), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:59: in parse
    result = handler.parse(source, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:41: in parse
    return self.process_context(ctx, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:59: in process_context
    self.parser.start(
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:95: in start
    child = item.child(qname, attrs, ns_map, len(objects))

        if self.config.fail_on_unknown_properties:
>           raise ParserError(f"Unknown property {self.meta.qname}:{qname}")
E           xsdata.exceptions.ParserError: Unknown property {http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType:{http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType

.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/nodes/element.py:469: ParserError

E           Failed: Failed to deserialize XML file 'tests/xml/ipxact/v1685_2022/Leon2/NXP.com/uart_scml/uart_scml.xml' using class '<class 'org.accellera.ipxact.v1685_2022.component.Component'>': Unknown property {http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType:{http://www.accellera.org/XMLSchema/IPXACT/1685-2022}portAccessType

Somehow portAccessType is not correctly bound to the correct class.

tefra · 2024-12-08T18:26:34Z

Confirmed something wrong is going on here, I am on it

tefra · 2024-12-08T18:49:53Z

Is there a place where I can download a bunch of these samples, something like a repo somewhere?

amal-khailtash · 2024-12-08T22:40:42Z

There are many examples here:

Matching these schemas:

Out of 618 xml files, only 19 are failing. I think both are related to portAccessType of Component and Abstractor classes/types.

FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/NXP.com/serial_device/serial_device.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/NXP.com/adaptor_scml/scmladaptor.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/NXP.com/uart_scml/uart_scml.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/uart_tac/1.0/uart_tac.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/pv2tac/1.0/pv2tac.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/apbbus/1.4/apbbus8.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/serial_device/1.0/serial_device.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/scmlAdaptor/1.0/scmlAdaptor.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2014 - xml/ipxact/v1685_2014/Leon2/spiritconsortium.org/Leon2TLM/uart_scml/1.0/uart_scml.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/uart_scml/1.0/uart_scml.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/pv2tac/1.0/pv2tac.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/uart_tac/1.0/uart_tac.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/serial_device/1.0/serial_device.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/apbbus/1.4/apbbus8.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/spiritconsortium.org/Leon2TLM/scmlAdaptor/1.0/scmlAdaptor.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/NXP.com/adaptor_scml/scmladaptor.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/NXP.com/uart_scml/uart_scml.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/Leon2/NXP.com/serial_device/serial_device.xml]
FAILED tests/test_xml.py::test_parse_xml[ipxact - v1685_2022 - xml/ipxact/v1685_2022/accellera.org/Sample/SampleComponent/1.0/xml/SampleComponent.xml]
========================================================================
19 failed, 618 passed, 25 warnings in 13.32s
========================================================================

tefra · 2024-12-09T15:40:07Z

Thanks for reporting @amal-khailtash very interesting suite, I took the liberty to include it in the samples repo https://github.com/tefra/xsdata-samples/tree/main/ipxact

The fix is on main!

amal-khailtash · 2024-12-09T16:07:03Z

That is great. All the XMLs' in those files pass now. I do have one more sample file:

SampleComponent.tar.gz

from a document that is NOT passing and failing with this:

.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:67: in from_string
    return self.from_bytes(source.encode(), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/mixins.py:88: in from_bytes
    return self.parse(io.BytesIO(source), clazz, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:58: in parse
    result = handler.parse(source, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:42: in parse
    return self.process_context(ctx, ns_map)
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/handlers/lxml.py:60: in process_context
    self.parser.start(
.venv/lib/python3.12/site-packages/xsdata/formats/dataclass/parsers/bases.py:94: in start
    child = item.child(qname, attrs, ns_map, len(objects))

E           Failed: Failed to deserialize XML file 'xml/ipxact/v1685_2022/accellera.org/Sample/SampleComponent/1.0/xml/SampleComponent.xml' using class '<class 'org.accellera.ipxact.v1685_2022.component.Component'>': Unknown property {http://www.accellera.org/XMLSchema/IPXACT/1685-2022}BaseAddresses:{http://www.accellera.org/XMLSchema/IPXACT/1685-2022}remapAddress

The problem is with this under Component type.

        <ipxact:busInterface>
            <ipxact:name>MirroredTarget</ipxact:name>
            <ipxact:busType vendor="accellera.org" library="Sample" name="SampleBusDefinition" version="1.0"/>
            <!-- LINK: mirroredTarget: see 6.7.7, Mirrored target interface -->
            <ipxact:mirroredTarget>
                <ipxact:baseAddresses>
                    <ipxact:remapAddress>'h40000000</ipxact:remapAddress>
                    <ipxact:range>'h1000</ipxact:range>
                </ipxact:baseAddresses>
            </ipxact:mirroredTarget>
        </ipxact:busInterface>

tefra · 2024-12-09T16:40:09Z

That sample doesn't pass the schema validation, doubled checked with lxml schema validator

tefra · 2024-12-09T16:41:19Z

I think you are missing a wrapper

amal-khailtash · 2024-12-09T18:18:57Z

Thanks I will check the XML file. You might be right.

amal-khailtash · 2024-12-09T21:24:51Z

Confirmed. Thank you!

tefra closed this as completed Dec 8, 2024

tefra reopened this Dec 8, 2024

tefra added the bug Something isn't working label Dec 8, 2024

tefra mentioned this issue Dec 9, 2024

Fix: Priorities when selecting source attr type #1107

Merged

4 tasks

tefra closed this as completed in #1107 Dec 9, 2024

amal-khailtash added a commit to amal-khailtash/pyipxact that referenced this issue Dec 9, 2024

Regenerated bindings after tefra/xsdata#1106 bugfix

d276cca

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xsdata creates two classes for the same type #1106

xsdata creates two classes for the same type #1106

amal-khailtash commented Dec 8, 2024

tefra commented Dec 8, 2024 •

edited

Loading

amal-khailtash commented Dec 8, 2024

amal-khailtash commented Dec 8, 2024 •

edited

Loading

tefra commented Dec 8, 2024

tefra commented Dec 8, 2024

amal-khailtash commented Dec 8, 2024 •

edited

Loading

tefra commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

tefra commented Dec 9, 2024

tefra commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

xsdata creates two classes for the same type #1106

xsdata creates two classes for the same type #1106

Comments

amal-khailtash commented Dec 8, 2024

tefra commented Dec 8, 2024 • edited Loading

amal-khailtash commented Dec 8, 2024

amal-khailtash commented Dec 8, 2024 • edited Loading

tefra commented Dec 8, 2024

tefra commented Dec 8, 2024

amal-khailtash commented Dec 8, 2024 • edited Loading

tefra commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

tefra commented Dec 9, 2024

tefra commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

amal-khailtash commented Dec 9, 2024

tefra commented Dec 8, 2024 •

edited

Loading

amal-khailtash commented Dec 8, 2024 •

edited

Loading

amal-khailtash commented Dec 8, 2024 •

edited

Loading