Created 2 commands to process gitlab logs
These commands are useful if you want to process gitlab-ci logs.
For instance,
bdt gitlab get-pipelines bob/bob.nightliesgets all the recent pipelines from bob/bob.nightlies. Something like this
Jobs from project bob/bob.nightlies
-------- ------------- -------- --------------------------------------------------------
Pipeline Branch Status Jobs
34568 big-issue-106 failed [178106, 178107, 178108]
34567 master success [178100, 178101, 178102, 178103, 178104, 178105]Finally,
bdt gitlab process-pipelines bob/bob.nightlies 34567 --job-id 178101summarizes execution time of a particular pipeline job. If nightlies is set, the stdout is split by project and we have an output like the one below.
Pipeline 34567, Job 178101
----------------------------------------------- ------------------- -------
None 2019-10-20 19:19:37 0.5m
bob/bob.buildout@master (1/120 2019-10-20 19:20:08 0.97m
bob/bob.extension@master (2/120 2019-10-20 19:21:07 2.35m
bob/bob.blitz@master (3/120 2019-10-20 19:23:28 11.7m
bob/bob.core@master (4/120 2019-10-20 19:35:10 15.25m
bob/bob.io.base@master (5/120 2019-10-20 19:50:25 16.57m
bob/bob.math@master (6/120 2019-10-20 20:06:59 13.68m
bob/bob.measure@master (7/120 2019-10-20 20:20:40 31.5m
bob/bob.io.image@master (8/120 2019-10-20 20:52:10 20.9m
bob/bob.db.base@master (9/120 2019-10-20 21:13:04 1.2m
bob/bob.io.video@master (10/120 2019-10-20 21:14:16 18.03m
bob/bob.io.matlab@master (11/120 2019-10-20 21:32:18 12.57m
bob/bob.io.audio@master (12/120 2019-10-20 21:44:52 15.85m
bob/bob.sp@master (13/120 2019-10-20 22:00:43 13.7m
bob/bob.ap@master (14/120 2019-10-20 22:14:25 14.55m
bob/bob.ip.base@master (15/120 2019-10-20 22:28:58 22.72m
bob/bob.ip.color@master (16/120 2019-10-20 22:51:41 13.87m
bob/bob.ip.draw@master (17/120 2019-10-20 23:05:33 10.55m
bob/bob.ip.gabor@master (18/120 2019-10-20 23:16:06 37.2m
bob/bob.learn.activation@master (19/120 2019-10-20 23:53:18 15.4m
bob/bob.learn.libsvm@master (20/120 2019-10-21 00:08:42 16.27m
bob/bob.learn.linear@master (21/120 2019-10-21 00:24:58 19.0m
bob/bob.learn.mlp@master (22/120 2019-10-21 00:43:58 16.87m
bob/bob.learn.boosting@master (23/120 2019-10-21 01:00:50 16.17m
bob/bob.db.iris@master (24/120 2019-10-21 01:17:00 1.42m
bob/bob.learn.em@master (25/120 2019-10-21 01:18:25 24.52m
bob/bob.db.wine@master (26/120 2019-10-21 01:42:56 0.95m
bob/bob.db.mnist@master (27/120 2019-10-21 01:43:53 1.05m
bob/bob.db.atnt@master (28/120 2019-10-21 01:44:56 1.1m
bob/bob.ip.facedetect@master (29/120 2019-10-21 01:46:02 15.72m
bob/bob.ip.optflow.hornschunck@master (30/120 2019-10-21 02:01:46 13.18m
bob/bob.ip.optflow.liu@master (31/120 2019-10-21 02:14:57 15.88m
bob/bob.ip.flandmark@master (32/120 2019-10-21 02:30:50 31.0m
bob/gridtk@master (33/120 2019-10-21 03:01:50 1.17m
bob/bob.ip.qualitymeasure@master (34/120 2019-10-21 03:03:00 17.63m
bob/bob.ip.skincolorfilter@master (35/120 2019-10-21 03:20:38 1.85m
bob/bob.ip.facelandmarks@master (36/120 2019-10-21 03:22:30 3.2m
bob/bob.ip.dlib@master (37/120 2019-10-21 03:25:43 2.37m
bob/bob.ip.mtcnn@master (38/120 2019-10-21 03:28:05 2.37m
bob/bob.db.arface@master (39/120 2019-10-21 03:30:27 1.27m
bob/bob.db.asvspoof@master (40/120 2019-10-21 03:31:43 3.55m
bob/bob.db.asvspoof2017@master (41/120 2019-10-21 03:35:16 0.87m
bob/bob.db.atvskeystroke@master (42/120 2019-10-21 03:36:09 0.88m
bob/bob.db.avspoof@master (43/120 2019-10-21 03:37:02 2.65m
bob/bob.db.banca@master (44/120 2019-10-21 03:39:41 1.67m
bob/bob.db.biosecure@master (45/120 2019-10-21 03:41:21 0.83m
bob/bob.db.biosecurid.face@master (46/120 2019-10-21 03:42:11 0.87m
bob/bob.db.casia_fasd@master (47/120 2019-10-21 03:43:04 0.97m
bob/bob.db.casme2@master (48/120 2019-10-21 03:44:02 0.85m
bob/bob.db.caspeal@master (49/120 2019-10-21 03:44:53 0.93m
bob/bob.db.cohface@master (50/120 2019-10-21 03:45:49 1.57m
bob/bob.db.frgc@master (51/120 2019-10-21 03:47:24 0.8m
bob/bob.db.gbu@master (52/120 2019-10-21 03:48:12 0.98m
bob/bob.db.hci_tagging@master (53/120 2019-10-21 03:49:11 6.9m
bob/bob.db.ijba@master (54/120 2019-10-21 03:56:07 3.17m
bob/bob.db.ijbc@master (55/120 2019-10-21 03:59:18 14.1m
bob/bob.db.kboc16@master (56/120 2019-10-21 04:13:29 0.9m
bob/bob.db.lfw@master (57/120 2019-10-21 04:14:23 2.77m
bob/bob.db.livdet2013@master (58/120 2019-10-21 04:17:09 0.8m
bob/bob.db.mobio@master (59/120 2019-10-21 04:17:57 1.7m
bob/bob.db.msu_mfsd_mod@master (60/120 2019-10-21 04:19:39 1.37m
bob/bob.db.multipie@master (61/120 2019-10-21 04:21:02 1.83m
bob/bob.db.nist_sre12@master (62/120 2019-10-21 04:22:52 7.33m
bob/bob.db.putvein@master (63/120 2019-10-21 04:30:13 0.97m
bob/bob.db.replay@master (64/120 2019-10-21 04:31:11 1.2m
bob/bob.db.replaymobile@master (65/120 2019-10-21 04:32:23 1.48m
bob/bob.db.scface@master (66/120 2019-10-21 04:33:52 0.93m
bob/bob.db.utfvp@master (67/120 2019-10-21 04:34:48 3.92m
bob/bob.db.verafinger@master (68/120 2019-10-21 04:38:43 1.67m
bob/bob.db.fv3d@master (69/120 2019-10-21 04:40:23 1.98m
bob/bob.db.hkpu@master (70/120 2019-10-21 04:42:22 1.7m
bob/bob.db.thufvdt@master (71/120 2019-10-21 04:44:04 1.97m
bob/bob.db.mmcbnu6k@master (72/120 2019-10-21 04:46:02 2.27m
bob/bob.db.hmtvein@master (73/120 2019-10-21 04:48:18 1.72m
bob/bob.db.voicepa@master (74/120 2019-10-21 04:50:01 6.37m
bob/bob.db.xm2vts@master (75/120 2019-10-21 04:56:23 1.03m
bob/bob.db.youtube@master (76/120 2019-10-21 04:57:26 1.8m
bob/bob.db.pericrosseye@master (77/120 2019-10-21 04:59:15 0.98m
bob/bob.db.maskattack@master (78/120 2019-10-21 05:00:14 1.18m
bob/bob.db.casiasurf@master (79/120 2019-10-21 05:01:25 1.48m
bob/bob.db.fargo@master (80/120 2019-10-21 05:02:54 1.2m
bob/bob.bio.base@master (81/120 2019-10-21 05:04:06 3.98m
bob/bob.bio.gmm@master (82/120 2019-10-21 05:08:05 8.3m
bob/bob.bio.face@master (83/120 2019-10-21 05:16:23 4.97m
bob/bob.bio.spear@master (84/120 2019-10-21 05:21:21 2.88m
bob/bob.bio.video@master (85/120 2019-10-21 05:24:14 2.35m
bob/bob.bio.vein@master (86/120 2019-10-21 05:26:35 4.87m
bob/bob.db.voxforge@master (87/120 2019-10-21 05:31:27 1.42m
bob/bob.rppg.base@master (88/120 2019-10-21 05:32:52 1.57m
bob/bob.pad.base@master (89/120 2019-10-21 05:34:26 2.92m
bob/bob.pad.face@master (90/120 2019-10-21 05:37:21 6.42m
bob/bob.pad.voice@master (91/120 2019-10-21 05:43:46 1.73m
bob/bob.pad.vein@master (92/120 2019-10-21 05:45:30 1.55m
bob/bob.fusion.base@master (93/120 2019-10-21 05:47:03 1.67m
bob/bob.db.oulunpu@master (94/120 2019-10-21 05:48:43 2.13m
bob/bob.db.uvad@master (95/120 2019-10-21 05:50:51 1.78m
bob/bob.db.swan@master (96/120 2019-10-21 05:52:39 2.33m
bob/bob.db.cuhk_cufs@master (97/120 2019-10-21 05:54:59 2.05m
bob/bob.db.cbsr_nir_vis_2@master (98/120 2019-10-21 05:57:02 1.93m
bob/bob.db.nivl@master (99/120 2019-10-21 05:58:58 4.2m
bob/bob.db.pola_thermal@master (100/120 2019-10-21 06:03:10 2.85m
bob/bob.db.cuhk_cufsf@master (101/120 2019-10-21 06:06:01 1.4m
bob/bob.db.ldhf@master (102/120 2019-10-21 06:07:25 1.43m
bob/bob.ip.tensorflow_extractor@master (103/120 2019-10-21 06:08:51 3.48m
bob/bob.ip.caffe_extractor@master (104/120 2019-10-21 06:12:20 2.13m
bob/bob.bio.caffe_face@master (105/120 2019-10-21 06:14:28 2.2m
bob/bob.learn.tensorflow@master (106/120 2019-10-21 06:16:40 134.88m
bob/bob.learn.pytorch@master (107/120 2019-10-21 08:31:33 10.98m
bob/bob.bio.face_ongoing@master (108/120 2019-10-21 08:42:32 2.27m
bob/bob.bio.htface@master (109/120 2019-10-21 08:44:48 105.23m
bob/bob.db.drive@master (110/120 2019-10-21 10:30:02 1.02m
bob/bob.db.stare@master (111/120 2019-10-21 10:31:03 0.83m
bob/bob.db.chasedb1@master (112/120 2019-10-21 10:31:53 0.82m
bob/bob.db.iostar@master (113/120 2019-10-21 10:32:42 0.83m
bob/bob.db.hrf@master (114/120 2019-10-21 10:33:32 0.83m
bob/bob.db.rimoner3@master (115/120 2019-10-21 10:34:22 0.83m
bob/bob.db.drionsdb@master (116/120 2019-10-21 10:35:12 0.83m
bob/bob.db.refuge@master (117/120 2019-10-21 10:36:02 0.87m
bob/bob.db.drishtigs1@master (118/120 2019-10-21 10:36:55 0.82m
bob/bob.ip.binseg@master (119/120 2019-10-21 10:37:44 1.42m
bob/bob@master (120/120 2019-10-21 10:39:09 284.9m
----------------------------------------------- ------------------- -------