The Exam Developer, Partner Enablement, manages all aspects of the development process required to assess the knowledge, skills and abilities of Adobe Partners who implement solutions using the Adobe Marketing Cloud. The Exam Developer adheres strictly to industry standards for creating legally defensible, valid and reliable test vehicles. In this role you will collaborate and coordinate with multiple teams of Subject Matter Experts (SMEs) and stakeholders within Adobe to create appropriate assessment tools.
Responsibilities
- Conduct local Job Task Analysis (JTA); global JTA as needed.
- Identify and document appropriate knowledge, skills, and abilities needed to create exam objectives.
- Manage blueprint process and create test definition document outlining weighting of exam items per objective.
- Teach Subject Matter Experts (SME) how to write test questions that are psychometrically valid and map to the appropriate cognitive level based on exam and objective requirements.
- Revise and edit test questions to meet psychometric grammar requirements.
- Conduct and manage technical reviews to ensure the technical accuracy of test questions and ensure congruency with exam objectives.
- Ensure beta exams are conducted appropriately; provide guidance and identify profile for ideal candidates to participate in exam betas.
- Coordinate statistical analysis (working with Psychometric vendor) of all exam data, and determine passing score and final item selection for exams.
- Manage assigned development tasks per schedule; communicate issues on a timely basis to ensure adherence to delivery dates.
- Update exam schedules using Microsoft Project or other project management software.
- Follow established guidelines and procedures to ensure security of all test items.
- Prepare all test documents using appropriate tools such as item-banking software.
- Coordinate meetings (typically virtual, worldwide) and collaborate with SMEs and stakeholders across Adobe to ensure exam content is created accurately.
- Work with certification team to continually add to knowledge and processes that support enhancing Adobe's exam development process.
- Guide and manage psychometric vendors as needed to support Adobe's exam development needs
Requirements
- 4+ years experience managing exam development, or similar project management experience.
- Solid understanding of the exam development process as defined by the American Psychological Association.
- Experience conducting JTA/objective development meetings
- Experience writing and revising multiple-choice test questions
- Demonstrated ability to learn quickly, be a team player, and manage change effectively
- Must be comfortable coordinating and driving virtual team of SMEs to meet development schedules
- Bachelor's or Master's degree in Industrial and Organization Psychology, Psychometrics, Educational Assessment, Instructional Design, or equivalent work experience
Special consideration given for:
- Experience using Microsoft Project or OmniPlan
- Experience creating test questions that measure soft skills
- Experience creating certification and assessments for the IT/Software industry
- Experience using an item-authoring tool such as Item Workshop, CERTS, or Benchmark.
- Experience conducting standard setting meetings, including Modified Anghoff and Hofstee methods
