Process Troubleshooting using Patterns

The variability in your process output is caused by variability from your process inputs.

This means that patterns you observe in your process output (as measured by your key performance indicators, or KPIs) are caused by patterns in your process inputs.

Recognizing which pattern you're dealing with can, hopefully, lead you quickly to the source of variability so you can eliminate it.

Stable

Boring processes that do the same thing day in and day out are stable processes. Everyday you show up for work and the process is doing exactly what you expected. Control charts of your KPIs look like this:


Boring is good: it is predictable, you can count on it (like Maytag products) so you can plan around it. Well-defined, well-understood, well-controlled processes often take this form. The only thing you really have to worry about is job security (like the Maytag repairman).

Periodic

Processes where special-cause signals show up at a fixed interval exhibit a "periodic" pattern.


This pattern is extremely common because in reality, many things in life are periodic:

• Every day is a cycle.
• Manufacturing shift structures repeat every 7-days.
• The rotation of equipment being used is cyclical
• Plant run-rates

On one occasion, we had a rash of production bioreactor contaminations. By the end of it all, we had five contaminations over the course of seven weeks and they all happened late Sunday/early Monday. On Fridays going into the weekend, people would bet whether or not we'd see something by Monday of the following week. Here, the frequency is once-per-week and ultimately, the root cause was found to be related to manufacturing shifts, which cycle once-per-week.
All these naturally occurring cycles at varying intervals and the key to solving a the periodic pattern is identifying the periodic process input that cycles at the same frequency.

Step-change

A step-change pattern is when, one day, your process output changes and doesn't go back to the way it was... not exactly "irreversible", but at least "difficult to go back."


Step patterns are also commonly observed in manufacturing because many manufacturing activities, "can't be taken back." For example:

• After a plant shutdown when projects get implemented.
• After equipment maintenance.
• When the current lot of material is depleted and a new lot is used.

One time coming out of shutdown, we had a rash of contamination: every single 500L* bioreactor came down contaminated. It turns out that a project securing the media filter executed during changeover for safety reasons changed the system mechanics and caused the media filter to be shaken loose. Filter stability was restored with another project so that the safety modifications would remain.

Step pattern is harder to troubleshoot than the periodic pattern because the irreversibility makes the system untestable. The key to solving a step pattern is to focus on "irreversible changes" of process inputs that happen prior to the observed step change.

Sporadic

A sporadic pattern is basically a random pattern.


Sporadic patterns are unpredictable and difficult to troubleshoot because at its core, the special-causes in-process outputs are often caused by two or more process inputs coming together. When two or more process inputs combine to cause a different result than if either two inputs alone, this is called an interaction.

A good example is the Ford Explorer/Firestone tires debacle that happened in the early 2000's. At the time, they observed a higher frequency of Ford Explorer SUVs rolling over than other SUVs. After further investigation, the rolled-over Ford Explorers had tires mainly made by Firestone. Ford Explorer owners using other tires weren't rolling over. Other SUV drivers using Firestone tires weren't rolling over. It was only when Firestone tires AND Ford Explorers used in combination that caused the failure.

To be blunt, troubleshooting sporadic patterns basically sucks. The best thing about a sporadic pattern is that it tells you is to look for more complex interactions within your process inputs.

Summary

Because the categories of patterns are not well defined (i.e. "I know it when I see it"), identifying the pattern is subject to debate. But know that the true root cause of the pattern must - itself - have the same pattern.

Cell Size and Scale

Cell culture and fermentation scientists/engineers deal with the size of things they cannot see. Here's a cool website I came across that helps with the visualization.


http://learn.genetics.utah.edu/content/begin/cells/scale/
It uses Flash - not HTML5 - but it's worth about 30 seconds of your time.

As you zoom in, you'll see an antibody in there; it's crazy how small it is.

FDA Releases Draft Guidance on Biosimilars

A week ago, the FDA released three documents to comply with the Biologics Price Competition and Innovation Act (BPCI Act) to create competition within the biologics space.

1. Q&A for BPCI Act
2. Scientific Considerations of Biosimilarity
3. Quality Considerations of Biosimilarity

Why is the bankrupt U.S. government passing more laws to empower the FDA on this matter? Basically because the markets created biologics faster than the FDA was able to respond and this is essentially catch-up.

You see, unlike small biological molecules, large biological molecules (called "biologics") cannot be feasibly or economically synthesized with chemical reactions. Biotech companies differ from pharma companies because they genetically engineer microbes to secrete the complex biological molecules and then produce a lot of it with fermentation or cell culture.

The FDA has long held:

Quality cannot be tested into the product

I was at this IBC Conference once where a professor got up and said, "No one knows this to be more true than academia... each year we test our students more and more, and each year they don't get any smarter."

It has long been insufficient for drug companies to produce a drug that meets product quality specifications when substitute raw materials were used or procedures not followed during the manufacture of the drug.

Applying this rule to biologics, it would hold that anyone who doesn't have the original cell line used to manufacture the name-brand biologic would violate this FDA dogma and thus be forbidden from selling their biologic in the US markets.

In essence, there's no way for the FDA to allow biosimilars into the US markets without throwing everything they've been doing out the window. This lack of a regulatory pathway forbids biologics made by companies other than the original manufacturer to be sold in the U.S, thereby handing US biotechs monopolistic power.

This is why there's a BPCI Act in the first place. It is to force the FDA to create a pathway for generic biologics to enter the US markets and induce competition.

Watching the FDA on biosimilars has become a spectator sport for the wonkish Regulator Affairs folk. The folks over at Bioprocess Blog cover this much more thoroughly.

Suggested reading:

• FDA Takes Big Step Forward in Biosimilar Regulatory Pathway
• FDA.gov's Biosimilar Resource Page

Data Visualization of Bach's Cello Suite No. 1 - Prelude

If you have about a minute and earphones, here's a really cool HTML5 website you ought to visit:

baroque.me

For the non-classical-musically minded, this is what the Naturalist guy in Master and Commander: The Far Side of the World was playing on his cello when they made it to the Galapagos Island.

There are a couple of cool things going on here:

1. I can play this piece on the viola. I serenaded 3 girls at university with this piece.
2. It's written in HTML5 - the future of web apps.
3. It takes music played on 4 strings and 2 bow strokes and spreads it out to 8 strings and 4 points on rotating balls... taking a few complex processes and making it simple to visualize.

So basically, it combines a few personal and professional ideas and mashes it into something cool.

Pick Actionable Factors for Multivariate Analysis

Here's you:

• You collect a ton of data from your large-scale cell culture/fermentation process.
• You're going blind alt-tabbing between Excel and JMP.
• You spend waaayyyyyyy too much time pushing around data and not getting answers.

And when you finally have the data the way you want it, your multivariate analysis tells you something like,

Final NH4+ (mmol) and Peak Lactate (g/L) correlate with Volumetric Productivity (mg/L/day).

Scientific curiosities are great for long-term process understanding, but when you're in the middle of a flagging campaign, manufacturing managers want to hear about immediate and short-term actions they can take to meet the campaign goals.

The key to avoiding this career blunder (of presenting irrelevant work to your customers) is to select only actionable parameters for your main effects and interactions when building your multivariate analysis. In JMP, it looks something like:



In the above example, we can control inoculation density (Ini VCD) by extending the previous culture's duration. As well, a biologics license agreement may allow a window for executing pH shifts (VCD at pH Shift) as well when to feed (Cult Dur at Batch Feed). Actions that manufacturing can take by simple scheduling changes are ideal for putting into the multivariate analysis that deliver immediate solutions.

Constructing the main effects of your model by selecting actionable parameters is best for solving REAL manufacturing problems as well as for advancing your career as the person who finds the way to meet campaign goals.