Error de índice no definido sobre $ em-> clear() en Symfony2

Question

He escrito un comando de Symfony para importar algunos datos de una API. Funciona, pero el problema es que mi uso de memoria PHP aumenta cuando inserto un JSON grande en mi base de datos. Y mi unitOfWork aumenta en '2' después de cada importación de actividad.

que tienen ya no se ha establecido todos mis objetos usados, y he leído la documentación de Symfony2 cuando se quiere hacer masiva de lote: http://www.doctrine-project.org/blog/doctrine2-batch-processing.html

Pero cuando se utiliza $em->clear() mi gestor de la entidad da este error:

Notice: Undefined index: 000000007b56ea7100000000e366c259 in path-to-application\vendor\doctrine\lib\Doctrine\ORM\UnitOfWork.php line 2228

Aquí está mi código completo:

/** 
* @see Command 
*/ 
protected function configure() { 
    $this 
    ->setName('ks:user:runkeepersync') 
    ->setDescription('Synchroniser les activités d\'un utilisateur runkeeper') 
    ->setDefinition(array(
     new InputArgument('access_token', InputArgument::REQUIRED, 'Access token'), 
    )) 
} 

/** 
* @see Command 
*/ 
protected function execute(InputInterface $input, OutputInterface $output) { 
    $accessToken = $input->getArgument('access_token'); 
    $em = $this->getContainer()->get('doctrine')->getEntityManager(); 
    $UserHasServices = $em->getRepository('KsUserBundle:UserHasServices')->findOneByToken($accessToken); 
    if (!is_object($UserHasServices)) { 
    echo "Impossible de trouver l'utilisateur qui possède le jeton ".$accessToken.""; 
    } 
    $user = $UserHasServices->getUser(); 
    $service = $UserHasServices->getService(); 
    echo "avant de requérir l'api : ".memory_get_usage()."\n"; 
    try { 
    $rkApi = $this->getContainer()->get('ks_user.runkeeper'); 
    $rkApi->setAccessToken($accessToken); 
    $activities = $rkApi->getFitnessActivities(0,25); 
    $nbParPages = 25; 
    $nomberActivitites = $activities->size; 
    $aActivities = $activities->items; 
    $nbPages = floor ($nomberActivitites/$nbParPages); 
    $aEndurance = array("Running", "Cycling", "Mountain Biking", "Walking", "Hiking", "Downhill Skiing", "Cross-Country Skiing", "Snowboarding", "Skating","Wheelchair", "Rowing", "Elliptical", "Other"); 
    $aEnduranceUnderWater = array("Swimming"); 
    $enduranceOnEarthType = $em->getRepository('KsActivityBundle:SportType')->findOneByLabel("endurance"); 
    if (!is_object($enduranceOnEarthType)) { 
     echo "Impossible de trouver le type de sport d'endurance"; 
    } 
    $enduranceUnderWaterType = $em->getRepository('KsActivityBundle:SportType')->findOneByLabel("endurance_under_water"); 
    if (!is_object($enduranceUnderWaterType)) { 
     echo "Impossible de trouver le type de sport d'endurance sous l'eau "; 
    } 
    echo "Après avoir récupéré 25 activités : ".memory_get_usage()."\n"; 
    $a = 0; 
    for($i=0;$i<=$nbPages;$i++){ 
     if($i!=0){ 
     $activities = $rkApi->getFitnessActivities($i,25); 
     $aActivities = $activities->items; 
     } 
     foreach ($aActivities as $activity) { 
     $a = $a+1; 
     $codeSport = $this->formatNameSport($activity->type); 
     $sport = $em->getRepository('KsActivityBundle:Sport')->findOneByCodeSport($codeSport); 
     if (!is_object($sport)) { 
      $sport = new \Ks\ActivityBundle\Entity\Sport(); 
      $sport->setLabel($codeSport); 
      $sport->setCodeSport($codeSport); 
      $sport->setSportType($enduranceOnEarthType); 
      $em->persist($sport); 
      $em->flush(); 
     } 
     $activityDetail = json_decode($rkApi->requestJSONHealthGraph($activity->uri)); 
     if(in_array($activity->type, $aEndurance)){ 
      $urlActivitieDetail = $activityDetail->activity; 
      $ActivitySessionEnduranceOnEarth = new \Ks\ActivityBundle\Entity\ActivitySessionEnduranceOnEarth($user); 
      isset($activity->total_distance)? $ActivitySessionEnduranceOnEarth->setDistance($activity->total_distance) : ""; 
      isset($activity->duration)? $ActivitySessionEnduranceOnEarth->setDuration($this->secondesToTimeDuration($activity->duration)) : ""; 
      isset($activity->start_time)? $ActivitySessionEnduranceOnEarth->setIssuedAt(new \DateTime($activity->start_time)) : ""; 
      $ActivitySessionEnduranceOnEarth->setModifiedAt(new \DateTime('Now')); 
      $ActivitySessionEnduranceOnEarth->setSport($sport); 
      isset($activityDetail->total_calories)? $ActivitySessionEnduranceOnEarth->setCalories($activityDetail->total_calories) : ""; 
      isset($activityDetail->climb)? $ActivitySessionEnduranceOnEarth->setElevationGain($activityDetail->climb) : ""; 
      $maxElevation = 0; 
      $minElevation = 10000; 
      if(isset($activityDetail->path)){ 
      foreach($activityDetail->path as $gpsPoint){ 
       if($gpsPoint->altitude > $maxElevation){ 
       $maxElevation = $gpsPoint->altitude; 
       } 
       if($gpsPoint->altitude < $minElevation){ 
       $minElevation = $gpsPoint->altitude; 
       } 
      } 
      $ActivitySessionEnduranceOnEarth->setElevationMin($minElevation); 
      $ActivitySessionEnduranceOnEarth->setElevationMax($maxElevation); 
      } 
      $em->persist($ActivitySessionEnduranceOnEarth); 
      $em->flush(); 
      //Pour chaque activité on a un identifiant relatif au service qu'on synchronise 
      $ActivityComeFromService = new \Ks\ActivityBundle\Entity\ActivityComeFromService(); 
      $ActivityComeFromService->setActivity($ActivitySessionEnduranceOnEarth); 
      $ActivityComeFromService->setService($service); 
      $ActivityComeFromService->setIdWebsiteActivityService($activity->uri); 
      $ActivityComeFromService->setSourceDetailsActivity($rkApi->requestJSONHealthGraph($activity->uri)); 
      $ActivityComeFromService->setTypeSource("JSON"); 
      $em->persist($ActivityComeFromService); 
      $em->flush(); 
      echo "Import de l'activite num ".$a." type :".$activity->type." effectue avec success \n"; 
      unset($ActivitySessionEnduranceOnEarth); 
      unset($ActivityComeFromService); 
      echo "UnitOFWOrk -> ".$em->getUnitOfWork()->size()."\n"; 
     } 
     if(in_array($activity->type, $aEnduranceUnderWater)){ 
      $ActivitySessionEnduranceUnderWater = new \Ks\ActivityBundle\Entity\ActivitySessionEnduranceUnderWater($user); 
      isset($activity->total_distance)? $ActivitySessionEnduranceUnderWater->setDistance($activity->total_distance) : ""; 
      isset($activity->duration)? $ActivitySessionEnduranceUnderWater->setDuration($this->secondesToTimeDuration($activity->duration)) : ""; 
      isset($activity->start_time) && !empty($activity->start_time)? $ActivitySessionEnduranceUnderWater->setIssuedAt(new \DateTime($activity->start_time)) : ""; 
      $ActivitySessionEnduranceUnderWater->setModifiedAt(new \DateTime('Now')); 
      $ActivitySessionEnduranceUnderWater->setSport($sport); 
      isset($activityDetail->total_calories)? $ActivitySessionEnduranceUnderWater->setCalories($activityDetail->total_calories) : ""; 
      isset($activityDetail->notes)? $ActivitySessionEnduranceUnderWater->setDescription($activityDetail->notes) : ""; 
      $em->persist($ActivitySessionEnduranceUnderWater); 
      $em->flush(); 
      $ActivityComeFromService = new \Ks\ActivityBundle\Entity\ActivityComeFromService(); 
      $ActivityComeFromService->setActivity($ActivitySessionEnduranceUnderWater); 
      $ActivityComeFromService->setService($service); 
      $ActivityComeFromService->setIdWebsiteActivityService($activity->uri); 
      $ActivityComeFromService->setSourceDetailsActivity($rkApi->requestJSONHealthGraph($activity->uri)); 
      $ActivityComeFromService->setTypeSource("JSON"); 
      $em->persist($ActivityComeFromService); 
      $em->flush(); 
      echo "Import de l'activité num ".$a." type :".$activity->type." effectué avec succès\n"; 
      unset($ActivitySessionEnduranceUnderWater); 
      unset($ActivityComeFromService); 
     } 
     echo "Après chaque activité : ".memory_get_usage()."\n"; 
     unset($sport); 
     unset($activityDetail); 
     $em->clear(); 
     } 
    } 
    } catch (\Exception $e) { 
    throw $e; 
    } 
} 

Gracias, @AdrienBrault. He probado con --env=prod --no-debug, y es cierto que consume menos memoria, pero la memoria aún aumenta. ¿Cómo puedo realmente borrar el administrador de la entidad? y estabilizar la memoria?

Answer 1

intente restablecer el gestor de entidades con:

$this->getContainer()->get('doctrine')->resetEntityManager(); 

y luego:

$em = $this->getContainer()->get('doctrine')->getEntityManager(); 

Answer 2

Symfony registra todas las consultas SQL en entorno de desarrollo, por lo que primero que hay que desactivarlo

// disable logger 
$em->getConnection()->getConfiguration()->setSQLLogger(null); 

Puede utilizar detectores de eventos en entidades, también puede aumentar el uso de la memoria. Puede desactivarlas al igual que

// remove all listeners 
foreach ($em->getEventManager()->getListeners() as $event => $listeners) { 
    foreach ($listeners as $listener) { 
     $em->getEventManager()->removeEventListener($event, $listener); 
    } 
} 

Retire unset de su código, no hay necesidad de ellos, a medida que el administrador de entidades clara cada etapa de su ciclo.

// save and clear 
$em->flush(); 
$em->getUnitOfWork()->clear(); 

Recuerde que la doctrina puede optimizar sus consultas, y mejorar la perfomance de consultas si agrupa en una sola flush. Entonces, la mejor práctica sería ejecutar flush una vez sobre algunas partes de sus datos. Por ejemplo:

// collect 100 entities and then save them 
if (($i % 100) == 0) { 
    $em->flush(); 
    $em->getUnitOfWork()->clear(); 
}