The Issue

While creating a build process in Gulp, I came across an interesting problem. The task which copied files from one location to another would complete, but then a subsequent task which injected the copied files into an HTML document would not successfully complete.

The Journey (tl;dr)

My first thought was to make sure that the tasks were properly dependent. According to the Gulp documentation, if the proper dependencies are configured, a dependent task will not fire until the task on which it is dependent is complete.1

Below is an example of the build tasks with the proper dependencies defined:


gulp.task('copy-files', function(done){
	
	// Copy the files from the source folders
	gulp.src('/path/to/file/to/copy.js')
		.pipe(gulp.dest('/folder/to/copy/into'));

	// Trigger the callback to signify that the task is done
	done();
};

gulp.task('inject-files', ['copy-files'], function(){

	var sources = gulp.src('/path/to/copied/files.js');

	var target = gulp.src('/path/to/template.html');
			
	target.pipe(inject(sources))
		.pipe(gulp.dest('/folder/to/write/updated/template');
};

With the dependencies in place, one would correctly assume the task that injects the files will not execute until the task that copies the files is complete. The issue persisted despite this configuration.

It was clear that the issue with the files not injecting was due to the copy-files task not completing despite the use of the done() callback. This is confirmed by running the inject-files task once more as a standalone task, in which case, the task would properly inject the copied files as they were already in the folder from which the task reads. If the files are first removed and the group of tasks run again, the injection will not work.

The only explanation that came to mind was that the gulp.dest() task was asynchronous. This would explain why the callback was getting triggered before the files were present on the disk.

My next step was to review the documentation to see if there was a method available to know when gulp.dest() was finished. Apart from a few additional configuration options there was no such method available. Then I reviewed the documentation for gulp.src() and discovered that the method “returns a stream of Vinyl files . . .”.3

The “stream” that the documentation was referring to are Node.JS Streams which in this case act as an abstraction of the underlying files that are being manipulated.4 What’s more is that “All streams are instances of EventEmitter.”5

Knowing that the stream emits an event means that the Node.JS Event on() method can be used to listen for the event of interest, in this case, when the files are completely copied.

The last remaining piece to the puzzle was to figure out which event to listen for. The Node.JS Stream documentation explains that a writable stream emits a “finish” event “when the end() method has been called, and all data has been flushed to the underlying system . . .” 7 This is useful information, but it is not known for certain that the gulp.dest() method will emit this event.

To be certain that the “finish” event is emitted focus must be given to the readableStream.pipe() method. The documentation states that the first parameter is a destination (a writable stream) and the second is a boolean used to determine if the end() method is called on the destination, which in turn emits the “finish” event.8 The default for the second option is true which proves that the gulp.dest() method will indeed emit the “finish” event.

The Conclusion

Armed with the fact that the gulp.dest() is an Node.JS writable stream and that it is also an EventEmitter a simple modification of the code ensures that the done() callback is only executed when the file copying is truly done.


gulp.task('copy-files', function(done){
	
	// Copy the files from the source folders
	gulp.src('/path/to/file/to/copy.js')
		.pipe(gulp.dest('/folder/to/copy/into').on('finish', function(){

			// Trigger the callback to signify that the task is done
			done();
		
		}));

};

gulp.task('inject-files', ['copy-files'], function(){

	var sources = gulp.src('/path/to/copied/files.js');

	var target = gulp.src('/path/to/template.html');
			
	target.pipe(inject(sources))
		.pipe(gulp.dest('/folder/to/write/updated/template');
};

References:

1 gulp API docs – Async task support

2 gulp.dest

3 gulp.src

4 Node.JS Stream

5 API for Stream Consumers

6 emitter.on

7 Event: ‘finish#’

8 readable.pipe