LBNL Culture Survey: Methodology Overview
![](https://www.google.com/images/icons/product/drive-32.png)
Survey Development Approach
Over the course of April through December 2023, the Lab's Culture Data Scientist engaged in the following activities to develop the LBNL Culture Survey:
Reviewed academic literature for validated survey instruments related to engagement and similar outcomes
Used validated questions (and will conduct further validation)
Checked for face validity via stakeholder meetings
Cross-referenced with existing surveys done by divisions or by UC Berkeley and past surveys
Chose comparable questions where possible
Used similar constructs to prior surveys, with some new types of questions related to learning processes
Held discussions about key priorities with Lab Directorate, ERGs, IDEA Chairs Council, divisional representatives from user facilities, HR Division Partners, and other stakeholder groups
Took best practices from division surveys and tested survey questions on diverse groups in the Lab
Sampling Strategy
This culture survey will go to all Lab employees, both represented and non-represented. It will not go to affiliates/contingent workers.
The higher number of employees who take this survey, the more likely the results will be representative of employee experiences across the Lab as a whole. There is always a risk that those who take the survey are systematically different than those who do not (for example, survey-takers may already feel a greater sense of community and so they are more willing to take time to fill out surveys).
To help alleviate this risk, we will also use weighted quota-based random sampling to have a representative sample, likely by Area, gender and possibly race depending on sample availability/ability to maintain confidentiality.
Survey Validation
We will validate survey results. In particular:
Confirmatory factor analysis (CFA) to test whether the factorial structure was consistent
Inter item correlation within a construct to consider adjusting questions
We will use this information in conjunction with the regression analyses to consider adjusting constructs
We did not do quantitative pre-survey validity testing because:
Survey constructs and most items have been validated in prior surveys.
Constructs overlap so we do not expect a validation to show distinct constructs.
We do not want to take out a large subsample of the Lab for a validation
There is strong interest in each specific question
Data Sources
We will merge data collected by the survey with other data sources to understand the relationships between the different cultural constructs and areas of workplace well-being that the survey measures. Only the Culture Data Scientist will have the employee-level information linking survey responses to these data sources, and will immediately delete individual names -- please see the Data Privacy page. All reporting will be aggregated and confidential. These will include:
Retention
Years at the Lab
Work mode (offsite, onsite, hybrid)
Seniority
Job type
Area and division
Gender
Race/ethnicity
Career or term
(Sexual orientation, disability, veteran status and education level depending on data quality)
Transparency is Essential.
Privacy & confidentiality of your survey data will be handled with the utmost care. Multiple protocols have been put in place to protect your data and ensure that the LBNL Culture Survey is a feedback channel you can trust.
LBNL Culture Survey: Outputs
Overview
Construct scores: Scores by construct will be shown using the average responses of the questions in that construct, scaled to be out of the same denominator.
Descriptive statistics: Average score of each construct will be shown by the different variables listed in data sources (such as by Area and years at the Lab). We will provide crosstabs to look at the interactions between variables (such as by gender and job type within Areas) to the extent that we can maintain confidentiality.
OLS regression: This will be used to show which constructs most drive the employee engagement (i.e., an outcome measure in the survey), including the variables listed in the Data Sources section above as covariates.
Benchmarks: We will consider available industry benchmarks and prior divisional surveys as points of reference. This survey will serve as a baseline to look at cultural change over time.
Heatmaps
Shows average score from
1: strongly disagree to
5: strongly agree
Heatmaps by construct will have individual questions for columns. Heatmaps by subgroup will have subgroups for columns. Could alternatively show:
percent agree or strongly agree
percent disagree or strongly disagree
Qualitative Data
Open-ended questions will be analyzed using grounded theory to inductively develop themes that emerge via a close reading of the responses. The responses will then be coded by theme. The themes will be organized according to the constructs so as to integrate with and enrich our understanding of the quantitative results. The themes will also be compared for key subgroups such as by race and gender
We will report on the prevalence of the themes so as to understand how representative they are - the goal is to provide representative responses as well as useful outlier feedback
We will make illustrative quotes by combining several similar quotes to retain confidentiality.