Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(function): Fix getTimestamp to parse fractional seconds correctly for Spark #12491

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

NEUpanning
Copy link
Contributor

@NEUpanning NEUpanning commented Feb 28, 2025

In getTimestamp function, the fractional seconds of "1970-01-01 00:00:00.2"
should be parsed as 2 milliseconds instead of 200 milliseconds to be consistent
with Spark.

Part of: #10354
Fixes: gluten issue

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 28, 2025
@NEUpanning
Copy link
Contributor Author

@rui-mo Could you help to take a look?

Copy link

netlify bot commented Feb 28, 2025

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 2145ac5
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/67c5a0558cb8a600086bcaad

@NEUpanning NEUpanning changed the title fix: Fix the getTimestamp function for proper parsing fractional Seconds fix(function): Fix getTimestamp function for proper parsing fractional Seconds Feb 28, 2025
@NEUpanning NEUpanning changed the title fix(function): Fix getTimestamp function for proper parsing fractional Seconds fix(function): Fix getTimestamp function for proper parsing fractional seconds Feb 28, 2025
@NEUpanning NEUpanning changed the title fix(function): Fix getTimestamp function for proper parsing fractional seconds fix(function): Fix getTimestamp to parse fractional seconds correctly Feb 28, 2025
@rui-mo rui-mo changed the title fix(function): Fix getTimestamp to parse fractional seconds correctly fix(function): Fix getTimestamp to parse fractional seconds correctly for Spark Mar 3, 2025
// Returns null for invalid datetime format when legacy date formatter is
// used.
enableLegacyFormatter();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why do we need this change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[VL] Result mimatch when to_timestamp with fraction of seconds
3 participants