AWS Glue has updated its Apache Spark infrastructure to support Apache Spark 2.4.3 (in addition to Apache Spark 2.2.1) for Glue scripts submitted on development endpoints. This enables you to take advantage of stability fixes and new features available in this version of Apache Spark.

from Recent Announcements https://aws.amazon.com/about-aws/whats-new/2019/09/aws-glue-supports-ability-to-test-glue-etl-scripts-on-development-endpoints/