Quality and Regulatory

GOOD REGULATORY PRACTICE (GRP):
Ensuring Quality Throughout the Regulatory
Process
DRUG INFORMATION ASSOCIATION
2nd Latin American Regulatory Conference
Mexican Pharmaceutical Market
Sandra Martínez-Díaz
Nov 2009
USD millons and % MS
. Mexican Pharmaceutical Market =
.
9
14,000 drug products
9
46,000 medical devices
Marketed drug products: Patent, Generics (with/ without BE)
Audited
Goverment
8,573
2,091
(69.7% )
Generics
700
( 9.0%)
76.7%
Total
Market
12,300
(17%)
23.3%
Public Hospitals
774.9
(6.3%)
Source: IMS
Sandra Martínez-Díaz
Nov 2009
Mexican Regulatory Actors
COFEPRIS
Health Secretariat
COFEMER
IMPI
IMSS / ISSSTE /
CCAYAC
EC / IRB
Regulatory
documents
Pharmacovigilance
Center
Mexican
Pharmacopeia
Law
Official
Guidance
Codes
(Reglamento)
Sandra Martínez-Díaz
Nov 2009
The Mexican Ministry of Health
COFEPRIS Challenges
DRUG PRODUCTS: QUALITY,
SAFETY, EFFICACY
ORGANIZATION,
INFRASTRUCTURE AND
CAPABILITIES
40 Agency rewiers
NDA, ANDA, IND, RENEWAL
HARMONIZATION PARADIGMS
COFEPRIS Timeframes (days)
New Drug Application (NDA) timeframes (2007-2009)
600
UCL=622.7
500
400
_
X=426.9
300
LCL=231.1
200
07
07
07
07
07
07
07
07
07
0
0
0
0
0
0
0
0
0
2
2
2
2
2
2
2
2
2
0/
1/
0/
9/
8/
8/
6/
4/
3/
1
1
1
0
0
0
0
0
0
/
/
/
/
/
/
/
/
/
25
26
15
27
20
14
20
18
30
submission date
Sandra Martínez-Díaz
Nov 2009
Dealing with COFEPRIS timeframes
Sandra Martínez-Díaz
Nov 2009
EC/IRB Timeframes
EC TIMEFRAMES
95% CI for the Mean
120
EC TIMEFRAMES
100
85.5556
80
60
41.2667
40
20
0
18
2002
2003
2004
2005
YEAR
2006
2007
2008
Sandra Martínez-Díaz
Nov 2009
Documents delivery timeframes
EC APPROVAL TO DOCUMENTS READY AT PHARMA
95% CI for the Mean
30
25
21.0488
Days
20
19.4127
15.7308
15
11.1667
10
7.8
5
0
2002
2003
2004
2005
YEAR
2006
2007
2008
Sandra Martínez-Díaz
Nov 2009
Documents review to RA Submission
DOCUMENTS REVIEW TO RA SUBMISSION
95% CI for the Mean
10
DAYS
8
6
5.45833
5.17391
4
2
0
2.63415
2.2
2002
2003
2004
1.74545
2005
YEAR
2006
2007
2008
Sandra Martínez-Díaz
Nov 2009
RA submission to Agency submission
(filing preparation)
RA Submission to Agency submission
95% CI for the Mean
7
6
Days
5
4.66667
4
3
2.93103
2.09524
2
1.47619
1
0
2002
2003
2004
2005
YEAR
2006
2007
1.22222
2008
Sandra Martínez-Díaz
Nov 2009
Mexican Agency timeframes
Mexican Agency timeframes
95% CI for the Mean
90
80
70
Days
60
52.875
50
45.2
40
40.3333
30
20
31.2857
28.9167
19.3438
16.75
10
0
2002
2003
2004
2005
YEAR
2009
2006
2007
2008
98 days +
7 days
• Inconsistent standards
• Lack of process ownership
• Deficient process management
2
98
7
Site Ready
1
Documents
ready
Doc review
EC Approval
PI (site)
86
Agency
approval
2
LPO
7
Filing prep
22
FPS
15
CTA
Submission
PA
Protocol
translation
Customization
Total CSA 247 days
7
Sandra Martínez-Díaz
Nov 2009
Hi Level Process
Country: Mexico * (this information is based on calendar dates)
Ensuring Regulatory Process Throughout
Quality
Clinical
Data
Quality
Site Quality Levels (Sigma values) distributon *
5.0
MONITOR
A LA TORRE
A RRIA GA
BRIONES
C ARDENAS
C ASTILLO
C ERVANTES
C ORONA
C RA VIOTO
C UEVAS
ECHARTEA
GUERRA
HERNANDEZ
LOPEZ
MOSQUEDA
PAREDES
PONTONES
REYES
4.5
Sigma Level
4.0
3.5
3.0
2.5
2.0
1.5
Ene-04
May-04
Sep-04
Ene-05
Date
May-05
Sep-05
Ene-06
Sandra Martínez-Díaz
Nov 2009
Is there any relationship between CRA Capabilities and data Quality ?
Sandra Martínez-Díaz
Nov 2009
Quality Tools and DMAIC
Mapping
Reducción de Verificación de documentos fuente
D
E
T
7
Falta de
especificacione
s claras
7
Existe un listado de
verificación que no ha
sido validado
1
Falta de
entrenamiento
8
8
Falta o
exceso de
revisión
(desperdicio)
5
Evaluación de la
productividad por
monitor con base
al número de datos
revisados (SDV)
FMEA
Falta de
planeación y
sobrecarga
de trabajo
8
Ninguno
1
No existe un
programa de
entrenamiento
que certifique la
calificación de
sitios
Falta de
herrramienta
basada en un
análisis
estadístico o de
tendencias
No existe un
método
confiable para la
evaluación de la
productividad
de los
monitores co
base a SDV
10
7
Existe un programa
general de
entrenamiento a los
sitios, mismo que dan
los monitores que no
han sido calificados.
Ninguno
10
10
1
1
Existe una base de
datos denominada
GTMS que pretende
calcular el número de
horas que los monitores
dedican a la actividad de
SDV, sin embargo esa
base no refleja la
cantidad de datos que
revisa cada monitor
1
R
P
N
Acciones
recomendadas
1) Normalizar la
operación mediante
la elaboración de
un SOP
1) Entrenamiento en
el SOP.
2) Verificación del
56 nivel de acuerdo
intra e inter
verificador
Resp.
1,5, 7, 12
Karin Mauer
2,5,7,12
49
1) Diseño de un
Programa de
entrenamiento en el
80
trabajo basado en
el expertise actual
de los monitores
1) Elaboración de
una herramienta
basada en Control
estadístico y en
AQL que permita
50 determinar la
cantdad de datos a
revisar en cada
visita basado en un
nivel de aceptación
de calidad
1) Diseño de una
herramienta que
permita visualizar la
carga e trabajo del
monitor con base al
80 SDV
2,5
1) Karin
Mauer
2) Sandra Mtz
3,6
3,4,6
6
1) Laura Ruy
2) Jorge Cruz
4
4
8,13
8,13
1) Sandra Mtz
8,13
9
10
Ranking
Total
Evidencia científica
(documentos
fuente/CRF)
DNF (papel, sistema)
Datos monitoreados
Reporte de Monitoreo
Variables (KPIV)
Instalaciones del sitio adecuadas
Equipo calibrado
Competencia del Inv
Competencia del equipo de
investigación del sitio
Carga alta de trabajo del monitor
Competencia del monitor
Número de datos revisados
Diseño del entrenamiento
Calificación del sitio
Tiempo de envío de datos
Velocidad de transmisión
Tiempo de captura
Diseño de queries para identificar
desviaciones
Competencia del coordinador para
identificar desviaciones
10
0
0
0
0
0
5
90
9
10
10
0
0
0
3
0
0
0
0
0
0
2
9
60
157
12
6
3
0
3
0
0
0
7
109
8
0
9
4
5
10
0
3
189
4
0
7
10
10
10
0
3
239
3
0
0
0
2
2
0
0
5
0
2
0
0
10
10
10
5
4
0
8
2
8
2
0
0
10
3
10
0
0
0
10
0
0
0
0
3
5
2
10
0
0
4
256
164
256
71
44
55
2
5
1
10
14
13
0
0
0
0
0
10
10
150
7
0
2
0
0
0
10
0
64
11
600
Controles actuales
7
165
Número de Revisión
discrecional al 25%
datos
revisados de los Dif. módulos
de los CRF
Calificación
inadecuada
del sitio
(Falso
positivo)
O
C
U
R
10
6
215
Entrenamiento en
preestudio
deficiente
Calificación
inadecuada
del sitio
(Falso
positivo)
Causa potencial
5
5
70
Falta de
experiencia en los
verificadores de la
calificación
Calificación
inadecuada
del sitio
2,5,13
S
E
V
5
4
531
Calificación Verificación de
del sitio
especificaciones
Efecto
2
3
175
Modo de falla
9
2
148
KPIV
7
1
Sitio entrenado
CTQ
Proceso
Versión: 13-Ene.2006
4
Sitio aprobado
Nivel de Importancia para el
cliente
Monitor a cargo
Nombre del Proyecto:
80%
70%
1) Julio
Cárdenas
Pareto chart
KPIV
Sandra Martínez-Díaz
Nov 2009
Variables
(Independent Variables
Site Qualification
Source document verification (Data Points)
CRA Competences/ Expertise
CRA Work-load
CTQ
(Dependent VARIABLE)
Y = Data Quality
Expressed as number defects per million opportunities
Quality
Measurement
System
Sandra Martínez-Díaz
Nov 2009
Relationship between data quality and study complexity
Residual Plots for ADJUSTED SIGMA LEVELS
Normal Probability Plot of the Residuals
Residuals Versus the Fitted Values
0.050
99.9
0.025
90
Residual
Percent
99
50
10
1
0.1
-0.050
-0.050
-0.025
0.000
0.025
Residual
0.050
0.240
0.050
15
0.025
10
5
0
0.252
0.264
Fitted Value
0.276
0.288
Residuals Versus the Order of the Data
20
Residual
Frequency
Histogram of the Residuals
Ho: Study complexity is not associated with Quality level
Ha: Study complexity is associated with Quality level
0.000
-0.025
0.000
-0.025
-0.050
-0.04
-0.02
0.00
Residual
0.02
0.04
1 5 10 15 20 25 30 35 40 45 50 55 60 65 70
Observation Order
„
„
„
„
„
„
„
„
„
„
„
Staff
Data Acquisition and Management
Clinical & La Supplies management
Regulatory aspects
Quality assurance
Facilities/ Infraestructure
Equipment
Site Management
Site Performance
Communication with sponsor
Total
Sandra Martínez-Díaz
Nov 2009
Site Qualification Tool
Sandra Martínez-Díaz
Nov 2009
DMAIC
Site Certification
Probability Plot of RESI6
Normal
99
Mean
StDev
N
AD
P-Value
95
90
0.0007882
0.01205
43
0.301
0.565
Percent
80
70
60
50
40
30
20
10
5
1
-0.03
-0.02
-0.01
0.00
0.01
RESI6
0.02
0.03
0.04
0.05
Residual Plots for ADJUSTED SIGMA LEVELS
Normal Probability Plot of the Residuals
Residuals Versus the Fitted Values
99
0.04
Residual
Percent
90
50
0.02
0.00
10
1
-0.02
-0.02
0.00
0.02
Residual
0.04
0.22
Histogram of the Residuals
0.30
0.04
12
Residual
Frequency
0.26
0.28
Fitted Value
Residuals Versus the Order of the Data
16
8
4
0
0.24
0.02
0.00
-0.02
-0.02 -0.01 0.00 0.01 0.02
Residual
0.03
0.04
1 5 10 15 20 25 30 35 40 45 50 55 60 65 70
Observation Order
CRA Certification
Kappa Analysis
Æ Protocol review
• Basic Questionnaire
• Four standard sections:
– Design, Inclusion and exclusion criteria, procedures, efficacy and
safety measurements
• New protocol for overall personnel.
Æ Complexity:
High.
Æ Participants:
CRA
Æ First Exercise:
March 16, 2006
• Agreement inter-CRA
Æ Second exercise: May 25, 2006
• Agreement intra-CRA
Æ Type of analysis: Kappa by attributes
Sandra Martínez-Díaz
Nov 2009
How can predict a process using a biased measurement system?
Sandra Martínez-Díaz
Nov 2009
Interpretation
% Kappa value
Interpretation
-100 to 59
Random agreement
≥ 60
Marginal. Improvement is required
≥ 70
Good
≥ 90
Excellent
Sandra Martínez-Díaz
Nov 2009
Accuracy results by CRA
Each CRA vs Standard
Overall acuracy
acuracy
Overall
Sandra Martínez-Díaz
Nov 2009
Inter agreement
agreement
Inter
Inter-agreement and overall accuracy results
BASIC QUESTIONNAIRE
Efficay &
&
Efficay
Safety
Safety
Procedures
Procedures
Exc/Incl
Exc/Incl
Sandra Martínez-Díaz
Nov 2009
Design
Design
Accuracy results between CRA
All appraisers vs Standard
ÆInter agreement analysis for CRA was performed
ÆKappa analysis reveled a random agreement between CRA
ÆAgreement results must be improved
• On the job program training need to be developed
ÆIntra-agreement analysis should be performed
ÆKappa analysis will be a key tool for site’s qualification
Sandra Martínez-Díaz
Nov 2009
Conclusions
„
„
„
„
„
„
„
„
„
„
„
Staff
Data Acquisition and Management
Clinical & La Supplies management
Regulatory aspects
Quality assurance
Facilities/ Infraestructure
Equipment
Site Management
Site Performance
Communication with sponsor
Total
Sandra Martínez-Díaz
Nov 2009
Site Qualification Tool
GOOD REGULATORY PRACTICE (GRP):
Ensuring Quality Throughout the Regulatory
Process
DRUG INFORMATION ASSOCIATION
2nd Latin American Regulatory Conference