Pros and Cons for Outsourcing Process Development

by Oliver Yu

I visited a contract development and manufacturing organization (CDMO) a few months ago.

What CDMOs do is they develop the process for you. And if you choose, they'll also execute the manufacturing process for clinical or cGMP material.

In short, if you're outsourcing either process development or manufacturing, CDMOs are the people you outsource to (pardon the preposition-ending sentence).

Dan Stark, currently senior director in the Global MSAT group at Genentech, once said (paraphrased):

The winners of the age of biotechnology will be the people who can translate research discoveries into commercial product and those who already have the infrastructure will have a head start.

What he's saying is if you're just a research outfit and you come up with new molecular entities, you're not in the position to extract the full value of that NME. Say you're IDEC pharmaceuticals and you come up with Rituximab... unless you can get the product into the hands of consumers, you're just a research outfit.

Likewise, if you are a contract manufacturer like Lonza where you have the manufacturing know-how and can produce clinical or commercial material, you also are not in the position to extract the full value of your know-how simply because you rely on someone else for your pipeline. In a lot of ways, you're a commodity (see Samsung getting into the biologics manufacturing space with Samsung Biologics).

Dan is saying that the people... the companies who can take a NME through the clinic... through the FDA approval process and be able to make the drug product are the ones who can extract the full value.

So why would anyone with a NME outsource the crown jewels of their business and go with external process development?

It turns out there are a lot of reasons.  To be sure that we got it right, Zymergi has collaborated with Dr. KC Carswell Ph.D of Carswell Bioprocess Consulting to produce a whitepaper on outsourcing process development.

If you're thinking about sending your process development into the hands of a 3rd-party, you definitely need to read the pros and cons of outsourcing.

Get FREE Whitepaper on Outsourcing PD

Boosting Speed of ZST

If you're here for the cell culture/biologics support/contamination/OSI PI commentary, go ahead and skip this post.

With Google's increasing emphasis on the speed at which pages load in the browser, users of the Zymergi SQL Tool (ZST) will have a chance to download the newest version which can decrease the page load time by 90%.

For enterprise users, this shouldn't be too big a deal since there is no expectation of fast pages when you're on a corporate portal or intranet. But when you're using ZST as an outward facing query tool, speed matters to SEO rankings.

The ZST process goes:

• Use the URL to determine database (db), query (qry) and data presentation (t)
• Execute queries against database.
• Transform the data into desired output.
• Send to user.

Pretty simple. But the problem is that we dip into the database every, single, time. If we saved the data to disk we wouldn't have to go to the database. And if we saved the output to disk, we could just pull straight from the output.

Here's the newest version:

• Use the URL to determine database (db), query (qry) and data presentation (t)
• Check to see if output file already exists, if so send this file to user... skip following steps.
• Check to see if data file exists, if so... skipi to the Transform step.
• Execute queries against database.
• Save result to data file.
• Transform the data into desired output.
• Save result to output file.
• Send to user.

It's easier to understand what's going on starting from the middle and work your way out. Here's the logic:

If we've already queried the database, why not save this data to disk?
And if we've already transformed the result, why not save the output to disk?

If the output file exists, no need to run the query and the transform, just send the output file.
If the output file doesn't exist, but the data file does, just run the transform and send the output.

Our tests are showing significant decreases in page load times.

Get ZST

Biologics Process Development is NOT Biologics Manufacturing Sciences

by Oliver Yu

I often run into folks who think that Process Development duties are the same as or similar to Manufacturing Sciences... that you can port the skills and knowledge of late-stage process development straight over to manufacturing support.

Other than the Manufacturing Sciences lab looking like a PD lab, not much else is the same.
But let's think about this.

Historically, success was achieved when PD defines a process that yields high titers and stable quality characteristics with very little focus on manufacturability.

There's actually a lot wrong with this. Firstly, titer ought not be the optimized parameter; the correct KPI to measure cell culture productivity is volumetric productivity. Secondly, a process is defined by identifying critical process parameters and proving the limits of their ranges. I've seen cases CPPs were defined so narrow that it is impossible to operate within the defined ranges. I've also seen cases where CPPs were not defined at all. When million-dollar batches get rejected over brief excursions outside of the process limits, people start rethinking their strategy.

Over time, some PD groups began listening to manufacturing (sciences) and began understanding that when 100% gains in volumetric productivity can be erased if the process could be successfully executed half the time. They began focusing on fully exploring the parameter space and locking down wide limits for higher process capability.

Some overachieving PD groups even studied excursions and intentionally perturbed cultures to prove that brief excursions outside of the operating limits did not impact product quality. When MSAT campaign monitoring groups had these types of processes, life was heaven.

A Manufacturing Sciences team's job is scientific support of the commercial biologics process.

• The process is already defined (by PD).
• Critical process parameters are already known (by PD).
• The proven acceptable range already specified (by PD).

The goal now is to characterize variability at large-scale, understand how KPIs respond to the measurable main effects and interactions and find the "sweet spot" that produces the most product in the least amount of time.   Oh, and when SHTF, be available to harness your formidable process experience and act to save the campaign from failing the campaign goals.

Most of this can happen with statistical analysis and SPC... bunch of chemical engineers with "fun" software programs.  Sometimes you need lab support to test your hypothesis. Sometimes the motions in this lab look the same as that of a PD lab, but the superficial similarity is about all the overlap there is.

Overall, the day-to-day tasks executed by folks in PD are (and should be) drastically different than the tasks executed by Manufacturing Sciences.

Related:

• 10 Ways You Suck at Cell Culture Support
• How Manufacturing Sciences Works
• Get Manufacturing Sciences Consulting

[-18000] Have not heard from this server (OSI PI)

You're reading this because:

• You are an automation engineer with (OSI) PI administration duties.
• You support a PI High-Availability (PI HA) system
• Your PI collective keeps going out-of-sync

This is what you see when you launch the PI Collective Manager:

This is a sporadic issue, with many possible causes; but, one cause is this tuning parameter:

Replication_SyncTimeoutPeriod

From OSIsoft Techsupport,

If the time it takes to read configuration changes exceeds the Replication_SyncTimeoutPeriod, the operation is aborted and the Secondary may get out of sync with the Primary. In this case, no amount of retrying by the Secondary will be successful because the configuration changes are dequeued after the initial request. Replication to that Secondary will halt until it gets re-initialized.

The default value for Replication_SyncTimeoutPeriod is 300 seconds (a.k.a. 5-minutes). So if, for some reason, your secondary server disconnects from the primary for 5-minutes, you need to re-initialize.

Re-initializing the secondary server is essentially a "copy-paste" of the primary PI server onto the secondary and can take several hours since it starts with a full backup of the primary PI server first. If you can avoid having to babysit a secondary re-initialization, you ought to change this tuning parameter: Replication_SyncTimeoutPeriod.  It can take up to 20-minutes for a server to reboot, so choose wisely.

p.s. - Thanks, Joy Wang for the support.

Automation Software: Pick OPC to be more Futureproof

by Oliver Yu

When scoping out automation projects, we commonly run into the tension of cost versus extensibility.

A system that is "futureproof" is one that can withstand changes to requirements or specifications that are determined at some later time. As with a lot of decisions in life, choosing a short-term gain often comes at the expense of the long-term.

So is the case when deciding to go with OPC at a greater initial cost. OPC stands for "OLE for Process Control" where "OLE" is Microsoft terminology that stands for Object Linking Embedding. (I guess that would make OPC short for "Object Linking Embedding for Process Control.")

As discussed here and here, OPC is the standard for communications between automation systems. OPC is maintained by the OPC Foundation… an industry consortium and is essentially what makes vendor agnosticism possible.

Vendor Agnosticism

Not having to believe in or commit to or be beholden to a single vendor.

For example, suppose you want to go with a Rockwell system. Rockwell has an embedded historian called, "FactoryTalk." Under the hood, FactoryTalk is OSIsoft PI. Side-by-side, they share the same folder structure, the same commands, the same services.

So the logical thing to do when setting up a corporate PI system is to buy a PItoPI interface and call it a day, right?

Not so fast.

Suppose the facility has a Building Automation System (BAS) and while out-of-scope for this project, the data from the BAS is expected to go into this corporate PI system. What happens then?

One option is to find that specific BAS vendor and see if OSIsoft sells a PI interface for this BAS vendor.

A second option is to see if this BAS vendor sells an OPC server for their data and if so purchase the OPC server and purchase the OPCtoPI interface and connect the two via OPC.

What happens if another control system enters the picture (suppose a Finesse/DeltaV system was purchased and commissioned)? The same questions have to get asked again (and re-answered).

Futureproofing in this case means choosing OPC and running every new system through this standard of communication, even at a greater cost to the initial system. In short-term, there is a larger budget to justify; in the long-term you eliminate future work (Design Reviews, meetings, revisiting past decisions…)

Life Pro Tip: Go Buy this Book: ProActive Selling by Skip Miller

by Oliver Yu

Zymergi was started by an engineer and a computer scientist. And to scientists and engineers, erudition comes in the form of books.

An early problem we had was sales... as in, we didn't have very much of it.

So the way we went about getting sales was to... hold your laughter... buy and read books.

I'm there at the book store and there are books on sales everywhere. The one that stuck out to me was this one: ProActive Selling: Control the Process - Win the Sale.

By education, I'm a chemical engineer. By trade, I'm a cell culture process engineer in that I troubleshot the commercial cell culture processes at biologics manufacturers. So this notion that sales was a process spoke to me; none of the other books were saying anything of the sort.

Skip Miller, the author and thought-leader, is saying that the process of selling is the exact same as the process of buying. And that good salespeople follow the steps of the process and help the customers through this process.

The book also talks about sales tactics... techniques... not so much Jedi-mind tricks as much as proven ways of saying things. Questions to ask, templates for how to start meetings and ways of moving the buyer along in the process.

The book was good enough that I googled him and found out that he's the owner of M3 Learning and runs a sales school here in the SF Bay Area. After I went, I sent my co-founder.  The entire experience was eye-opening.

You see, what most people do NOT realize is that they are in the business of sales. They're in the business of sales because selling is persuasion.  Selling is getting someone to change: to change what they doing before to the new, more valuable way that you're offering.

Job hunters are trying to persuade hiring managers that they are the best candidate. Engineers are trying to sell management to not cancel their projects. Middle management is trying to sell individual contributors on the business process and simultaneously trying to persuade upper management on increasing budget and headcount.

Look around: leaders are good at persuasion and a lot of them rise in to the top of organizations because they are great at selling.

Be you a technical worker bee, an "A" account executive or an fresh university graduate, you owe it to yourself to at least buy his book.

And if you find that this book rings of truth and want a more in-person coaching, then sign up for Skip Miller's Sales School in San Jose. It's a 2-day course for under $2K. There's one coming up next month: Oct 7 - 9th, 2013.

Sign Up For Sales School

A lot of you come to this blog to read about cell culture, data historian, bioreactor sterility advice.  This is actually better since you can apply it to virtually all aspects of your life, not just your wonkish cell culture stuff.

disclosure: other than winning a 2nd edition copy of his book on FB, I don't get a dime from Skip Miller, M3 Learning or his associates. Skip is one of those things I recommend to everyone.

Pareto Principle at work in Biotech

In the latest FDAzilla post, guest blogger and regulatory professional - Greg Hattoy - talks about 5 ways to prepare for a regulatory audit.

The first idea is that small things can build into big things and that management needs to have a pulse on what's going on. It isn't so much that there's something small out there that's wrong with your plant (like rust on the bottom of the fermentor) that that it becomes big (like your fermentor gets a hole in the bottom of it from the rust).

I've seen this in action where immediate- and short-term fixes are executed in a way that causes long-term maintenance chores. For example:

• The intended design of post-inoculation additions was among the suspect causes of bioreactor contamination.
• In the name of mitigating this potential risk via process of elimination, post-inoc additions were changed away from the intended design.
• True cause of microbial contamination was later deemed to be something else.
• Due to "don't fix what ain't broke" and heavy workload, the modified additions were never changed back.

You get enough of these decisions and a well-engineered plant encumbers itself and slips farther away from good engineering practices.

His second idea is that there are "Big Wins." This is absolutely consistent with the Pareto Principle where 80% of the effects can be explained by 20% of the causes, and though Greg states that it my fly in the face of the "small things add up" thesis, it actually doesn't. Here's why:

Every biotech professional from worker bee on up to the top of management has their list of 100 to-do's. Everybody is loaded down and their email inbox just looks like a constant scroll.

There actually aren't 100 different problems that need to be solved. It's actually something like 12 real problems. The majority of the effect/response (80%) is caused by a minority of the causes/factors (20%).

There are people who tackle this list according to priority: #1...done. #2...done. #3....done. And by the end of the day, the top 10 items get subtracted off the list (and another 10 get added).

Then, there are people who look at the list and go, "Gosh, #3, #7, #13, #49, #62, #85 are all related. If I spend time tackling the true root cause, I can hit six birds with one stone!" And when you go about solving problems in this manner, you can use the Pareto principle in your favor:

You can solve 80% of your problems by fixing 20% of what's wrong.

Obviously, there are times when you need to tackle the list sequentially. But this ought to be a minority of the time compared to tackling your to-do list strategically.

Critics of the sensible approach say that walking away from regulatory inspections has more to do with the personality of the inspector.  For that, there are ways of gathering regulatory intelligence.   But for the rest of us, following defensible plan is the most sensible approach for preparing for a regulatory audit.