Project 4
Global Client Employee Engagement Research Survey a Statistical Analysis
Project Snapshot
Project: Assess employee engagement for corporate client as part of data analysis consultancy assignment
Role: Research Consultant (graduate class assignment)
Responsibility: Design Survey, Assess 900+, Report Findings and Recommendations
Methods: Likert scale survey
Analysis: Analysis of variance (ANOVA), Regression Analysis
Tools: Qualtrics, SPSS Software
About the Organization
The client is an industry leader with over 45% market share in North America.
Problem Statement
Hundreds of research studies, including a meta-analysis by Gallup, shows that employee engagement predicts a variety of critical business outcomes, including productivity and profitability. Organizations such as [—] know that engaged employees are critical to their success.
Therefore, [Organization] wants to assess and find ways to impove employee engagement of its 1,300+ employees as it credits much of it’s success to its positive work culture.
Introduction
Employee Engagement
[ORGANIZATION] has routinely assessed employee engagement via a survey. This year there have been a few changes in the organization and is curious to understand how these may be affecting employee engagement. There are variables the client wants to keep but is wanting a new angle for this new survey.
My Role
As part of an assignment for my Research Methods course I acted as a research consuoltant to this client, working closley with another fellow classmate. We collaborated in this research project to improve the clients survey to uncover new employee engagement insights. We worked together to analyze the data and offered the client recommendations based on our findings.
Research Objectives
Assess Level of Employee Engagement
Determine Drivers of Engagement
Assess Employee Ratings of Those Drivers
Determine Critical Areas of Improvement
Procedure
Materials
Updated Survey
Categories (Variables of Interest)
Working closely with my classmate we took the client’s existing survey and made some changes. There are five core categories that [ORG] has used in all of their surveys in the past. We decided to keep these and refined the survey by adding other categories we thought were important.
Variable Additions
Reflecting on the work experiences of graduate students in the class, the following organizational issues were the selected additions:
Business Units, Leadership, Health & Wellness, Communication, Team
Additions: Meaningfulness, Physical Environment, Social Support, Learning & Development, Flexibility, Compensation & Benefits, Cafeteria, Diversity & Inclusion, Engagement, Change Management, Job Satisfaction, Intent to Leave*
*Primary Outcome Variables (POV). These are key, they’re the most important outcomes. All the other variables are important but they tend to lead or affect the POVs and these are the items the [ORGANIZATION] wants to look out for.
SPSS Software
SPSS Software was downloaded and utilized to analyze the data.
Methodology
Research Design
Survey
I like to start with a cognitive walk through before a usability test for various reasons. The insights gained from a cognitive walkthrough can inform the creation of more realistic and relevant task scenarios for the usability test. It helps ensure that the tasks align closely with real-world user needs and challenges. In addition, performing a cognitive walkthrough helps me understand the user's journey and context better. This understanding is crucial in designing tasks that are meaningful and representative of actual user behavior. The best part? It requires little time and effort and there’s no need for recruitment.
Responses
Recruitment
1,325 survey’s distributed via email and sharepoint
The survey remained open for a full month, employees received weekly reminder via email.
945 responses (70% Response rate)
Likert scale questions, employees were asked to rate items on a 5 point likert scale
Demographics
14% Female, 86% Male (consistent with the makeup of the male-dominated company)
The youngest respondent to take the survey was 22 years old while the oldest was 75 years old.
The mean age of all respondents was 42 years old.
Respondents were a part of 1 of these 17 business units.
Analysis
Prior to running the statistical analysis, data was cleaned by:
Removing cases deemed to be too incomplete for analysis
Pursued dataset for obvious errors such as extreme straight-lining (e.g., giving the same response for every item on the survey).
Statistical Analysis
Ran frequencies and bar charts on all variables.
Computed appropriate measures of central tendency (for example, means, modes, or medians) and variance (for example, standard deviation) on all variables.
Recoded any variables as necessary for consistency.
Determined which variables should be combined into indices, and determined the internal reliability of those groupings by computing Cronbach's alpha to ensure that they actually should be put together into indices.
Computed indices of key issues.
Ran means on all indices.
To get a solid understanding of the employee data we also looked at outcome variables beyond just engagement:
Using Regression Analysis we assessed
potential associations between age and the indices & other key outcome variables.
and determined the predictors of engagement through regression analysis.
Using ANOVA (Analysis of Variance) we assessed potential
gender differences in the indices & key outcome variables (engagement, job satisfaction, intent to leave).
tenure differences in the indices & other key outcome variables.
differences in engagement & other important variables by business unit.
Results
Indices and Internal Validity
The analysis focused on multiple drivers of engagement (Indices aka Categories), including
Team, Social, Safety, Management, Learn, Leader, Health, Flexibility, Environment, Engagement, Diversity, Compensation, Communication, Change, and Cafeteria.
All categories mentioned above had high internal reliability, except for Flexibility (Cronbach's alpha less than .70), which was excluded from analysis.
Assessing by Business Units
We choose six key questions for the deeper examination of business units. All of these were directly related to engagement.
I am always willing to give extra effort to help [ORG] succeed.
It would take a lot to leave [ORG].
My work makes an important contribution to [ORG]’s success.
I feel [ORG] provides me with an equal opportunity for advancement.
Team performance is acknowledged.
Changes are coming at a pace I can keep up with.
-
Employee job satisfaction: 8.03 out of 10 (SD: 1.57), Most employees (81.86%) rated satisfaction close to this score.
Employees feel safe (mean: 4.52, SD: .66).
Employees value productivity and quality (mean: 4.50, SD: 6.20).
There is low intent to leave Logan within the next year (mean: 1.37, SD: .90).
Cafeteria-related aspects received mostly neutral scores, including food prices (mean: 3.06, SD: 1.13) and food quality (mean: 3.31, SD: 1.07).
Ratings for cafeteria and credit union hours of operation were also neutral (cafeteria mean: 3.42, SD: 1.18; credit union mean: 3.55, SD: .72).
Change management received a neutral score (mean: 3.56)
-
-There were no significant differences between gender and any of the categories analyzed.
-Age did not appear to be a predictor of any engagement outcome variables.
Significant differences by tenure regarding simplicity of the organization's Life System and the convenience of credit union hours of operation.
Though these scores came back as statistically significant the actual mean difference is quite minor and suggesting there is no major area of concern.
“The [ORG] Life System is easy to understand.”
-Tenure groups that had the highest seniority at [ORG] (21+ years) and those that were relatively new to [ORG] (1-4 years) both scored similarly on the [ORG] Life System’s perceived simplicity (average: 3.99/5.00).
-The employees with 5-10 years of experience and 11-20 years of experience scored an average of 3.86/5.00.
“The credit union hours of operation are convenient.”
-Those with 21+ years of [ORG] experience believe the credit union hours of operations are slightly more convenient (average of 3.68/5.00) compared to all other tenure groups (average about 3.50/5.00).
-
Significant difference between the business unit and feelings of wanting to leave [ORGANIZATION] (p-value .00).
-The Strategic Business Management business unit felt very strongly about not leaving [ORGANIZATION] with an average of 4.64, while the ESM business unit felt neutral but leaned towards slightly agreeing to planning on leaving [ORGANIZATION] (average of 3.97).
Significant difference between the business units and the employees' feelings about their work playing an important part in [ORGANIZATION]’S success (p-value .01).
-The Unit Managers were very close to strongly agreeing with the statement, with the highest average of 4.73 while the Strategic Capital Management Business unit simply agreed with an average of 4.00.
Significant difference in the willingness to give extra effort to help [ORGANIZATION] succeed by business unit (p-value .00).
-The ESS business unit agreed with this statement with the lowest average of 4.32 while the Unit Managers business unit held the highest average closer to strongly agree at 4.80 out of 5.00.
Significant difference in acknowledgment of team performance by the management team and the business unit (p-value .00).
-The Supply Chain business unit feels strongly about the acknowledgment of team performance with an average of 4.68 while the ESS business unit was neutral with an average of 3.56.
Significant difference between how employees feel about keeping up with the pace in which change is coming and the business unit (p-value .00).
-The Unit Manager business unit and the ESM business unit feel mostly neutral about keeping up with change, Unit Managers averaging 3.27 and the ESM business unit averaging 3.39.
-The DC4 business unit and the Strategic Business Management business unit agreed more with the ability to keep up with changes.
-The DC4 Business Unit had an average of 3.99 and the Strategic Business Management business unit had an average of 4.00.
Significant difference in employees feeling as though they’re provided an equal opportunity to advance by business unit (p-value .00).
-The ESM business unit feels neutral about being provided an equal opportunity to advance with the lowest average of 3.45.
-The Unit Managers agreed more with the statement with the highest average of 4.67.
-
-77% of engagement is determined by social support, change, compensation & benefits, physical environment, and meaningful work.
-Key drivers of engagement with p-values: Team (.00), Safety (.02), Diversity (.31), Social (.00), Meaningful (.00), Compensation (.00), Environment (.00), Change (.00).
-All key drivers have a positive relationship to employee engagement, meaning as one of these averages increase so will the average of engagement
What are Indices?
Indices are summary scores that condense a lot of data into a single number. They are used to measure and compare concepts that can be quantified based on several related factors. Example below.
-
To assess the [ORGANIZATIONS] change management from the employee’s perspective we create a "Change Index" that takes into account variables like employee confidence in the organization’s direction, level of communication employees receive pertaining to changes taking place, employees ability to keep up with pace of change, and employees feelings towards change management efficiency.
Change Index condenses all ratings pertaining to the following survey questions into a single number that summarizes the employees ratings of the organization’s change management.
“I feel confident in the direction [ORGANIZATION] is heading.”
“I receive sufficient communication pertaining to the changes taking place.”
“Changes are coming at a pace that I can keep up with.”
“Change is managed effectively here.”
Findings
Engagement is High
Overall, results indicated that employee engagement at the organization is significantly high.
8 Drivers of Engagement
Drivers of Engagement in order: Meaningful work, change management, Compensation & benefits, physical environment, social support, team, health/safety, diversity & inclusion
77% of engagement is determined by: Meaningful work, change management, Compensation & benefits, physical environment, social support
Most Impactful Drivers of Engagement: Meaningful Work & Change Management
Highest Drivers of Engagement: Meaning Work & Change
Meaningful Work was the most impactful driver of engagement
Job Satisfaction is High, Intent to Leave is Low
Employee job satisfaction: 8.03 out of 10 (SD: 1.57), Most employees (81.86%) rated satisfaction close to this score.
There is low intent to leave the organization in the next year.
Lowest Satisfaction in Cafeteria, Change Management, and Credit Union
Lowest scores fell within “neutral” ranges
Employees do not feel strongly about either the cafeteria food prices nor food quality.
Cafeteria hours of operations ratings were unremarkable.
Credit union hours of operations were unremarkable.
Employees neither agree nor disagree with the statement that change is managed effectively at [ORGANIZATION].
Most Business Units Feel Neutral Regarding Change Management of the Organization
Change is one of the key drivers and most business units felt neutral about being able to keep up with change.
SBM and DC4 were the units who felt more strongly about change management at the organization.
Differences by Tenure are Minor
Analysis of tenure revealed minor differences in responses, particularly related to the simplicity of the organization's Life System and the convenience of credit union hours of operation. Though these scores came back as statistically significant the actual mean difference is quite minor and suggesting there is no major area of concern.
Management-Level Business Units Have Higher Ratings
We choose six key questions to focus on for the examination of business units. All chosen questions were related to engagement drivers. Strategic Management had higher average ratings compared to other units, which generally leaned closer to"neutral" ratings on the following areas:
Desire to Not Leave the Organization
Importance of Work for Organization's Success
Willingness to Give Extra Effort to Organization
Acknowledgment of Team Performance
Ability to Keep Up with Change
Equal Opportunity to Advance
Age and Gender Have No Impact
Both men and women responded to the survey questions similarly and age does not seem to be a predictor of any of the engagement outcome variables.
No significant differences were found between gender and any of the categories (Team, Communication, Safety, Management, Leader, Diversity, Learn, Social, Meaningful, Compensation, Environment, Cafeteria, MISC, Change, Job Satisfaction, Intent to Leave and Engagement).
No significant associations were found between age and any of the categories or key variables.
Recommendations
In conclusion, while [ORGANIZATION] currently enjoys notably high employee engagement levels there are actionable opportunities for further enhancement. I’ve broken the recommendations down into three categories Change Management, Advancement Opportunities, and Cafeteria.
Change Management Recommendations
Change is one of the key drivers and was also the category with the lowest ratings thus improving the scores of change management can further enhance engagement. So how can this be done?
Two Way Communication
Open a two-way dialogue between upper management and employees. For example, involve employees in change decisions by running one meeting per quarter with randomly selected employees who can offer their thoughts and opinion on the process and roll outs of the organization’s changes taking place, then ensure this feedback is reported back to the decision makers, or better yet have a decision maker run and/or attend these meetings.
Standardize
Standardization of processes can maintain organization and efficiency across the organization. This can begin with addressing the disparities encountered among units. For example, turn to the business units who had the highest most positive scores and use them as exemplar models. Figure out what is working for them and begin drafting guides, programs, processes, that can be distributed, taught, and/or implemented across all business units. That may help close gaps in business unit engagement ratings, for overall improvement.
Disparities to focus on:
-
Strategic Business Management Unit and DC4 Unit felt more strongly and positively about the change pace compared to other units. What are SBM and DC4 doing? close gap
-
Overall SBM felt pretty strong in Ability to Keep Up with Change, Equal Opportunity to Advance, Desire to Not Leave the Organization, Importance of Work for Organization's Success, Willingness to Give Extra Effort to Organization, Acknowledgment of Team Performance. look at what their doing. Open dialogue and conversation may inspire ideas on how to implement new programs, processes, or measures, using SBM as an example model… may help close gaps in business unit engagement ratings, for overall improvement.
Advancement Opportunities Recommendations
Ensuring that all business units, apart from Strategic Management and Strategic Capital Management, offer clear and equitable career advancement paths can boost both employee morale and overall engagement levels. Ideas to consider or look into:
Transparent Communication
Clearly communicate the criteria, expectations, and opportunities for career advancement within the organization. Provide employees with a clear understanding of the skills, experience, and performance levels required for each level of advancement.
Performance Management
Implement a robust performance management system that includes regular feedback, goal setting, and performance evaluations. This helps employees understand their strengths, areas for improvement, and the specific steps they can take to advance in their careers.
Internal Mobility
Encourage internal mobility by promoting job postings and opportunities within the organization. This allows employees to explore different roles and departments, fostering growth and advancement without having to leave the company.
Fair & Objective Evaluation
Ensure that career advancement decisions are based on fair and objective criteria. Implement a structured evaluation process that considers performance, skills, and potential rather than favoritism or bias.
Employee Support Programs
Establish programs that provide career guidance, coaching, and mentorship to employees. This helps them navigate their career paths, set goals, and receive guidance from experienced professionals within the organization.
Recognition & Rewards
Recognize and reward employees for their achievements and contributions. This can include promotions, salary increases, bonuses, or other forms of recognition that acknowledge their growth and advancement.
Cafeteria Experience Recommendations
Ratings surrounding the cafeteria did not have a significant impact on employee engagement but since overall cafeteria ratings were among the lowest, below are ideas to improve the food prices and food quality:
Food Service Partnership
Evaluate the possibility of partnering with a professional food service provider who can offer expertise in menu planning, sourcing quality ingredients, and managing costs. They can help enhance the overall food experience for employees.
Involve Employees
Involve employees in the decision-making process by forming a cafeteria committee or conducting focus groups. This way, employees can contribute ideas, preferences, and suggestions for improving the food offerings.
Discounts or Loyalty Programs
Consider implementing discount programs or loyalty rewards for employees to encourage their participation and make the cafeteria more appealing in terms of affordability.
